Setting up a Bootable EFI

In my setup I continued on my old post Multi-Boot PE using Grub4Dos. I did not re-formatted my HDD in GPT, it is still in MBR.
But before going to setup best to know what you are against.

What is EFI or Extensible Firmware Interface?
According to Wikipedia, the EFI System partition (ESP) is a partition on a data storage device (usually a hard disk drive or solid-state drive) that is used by computers adhering to the Unified Extensible Firmware Interface (UEFI).

When a computer is powered up and booted, UEFI firmware loads files stored on the ESP to start installed operating systems and various utilities. An ESP needs to be formatted with a file system whose specification is based on the FAT file system and maintained as part of the UEFI specification; therefore, the file system specification is independent from the original FAT specification.

ESP contains the boot loader programs for all installed operating systems (which are contained in other partitions on the same or other storage device), device driver files for devices present in a computer that are used by the firmware at boot time, system utility programs that are intended to be run before an operating system is booted, and data files such as error logs.


Also making the story short if you will read the Wikipedia UEFI can boot on MBR (thru CSM) and GPT. Also from my short study I found that to be able to boot on UEFI your primary partition needs the following file & folder structure...

(From rodsbooks)
The default/fallback boot loader, EFI/BOOT/bootx64.efi
The Windows boot loader, EFI/Microsoft/Boot/bootmgfw.efi
The OS X boot loader, System/Library/CoreServices/boot.efi (typically stored on the OS X system partition)
An EFI shell program, shellx64.efi, if it's stored in the root of the ESP


Also see "www.uefi.org/registry"

What is MBR?
MBR standards for Master Boot Record. It was introduced with IBM PC DOS 2.0 in 1983.
It’s called Master Boot Record because the MBR is a special boot sector located at the beginning of a drive. This sector contains a boot loader for the installed operating system and information about the drive’s logical partitions. The boot loader is a small bit of code that generally loads the larger boot loader from another partition on a drive. If you have Windows installed, the initial bits of the Windows boot loader reside here — that’s why you may have to repair your MBR if it’s overwritten and Windows won’t boot. If you have Linux installed, the GRUB boot loader will typically be located in the MBR.

MBR works with disks up to 2 TB in size, but it can’t handle disks with more than 2 TB of space. MBR also only supports up to four primary partitions — if you want more, you have to make one of your primary partitions an “extended partition” and create logical partitions inside it. This is a silly little hack and shouldn’t be necessary.

Hello GPT!
GPT stands for GUID Partition Table. It’s a new standard that’s gradually replacing MBR. It’s associated with UEFI — UEFI replaces the clunky old BIOS with something more modern, and GPT replaces the clunky old MBR partitioning system with something more modern. It’s called GUID Partition Table because every partition on your drive has a “globally unique identifier,” or GUID — a random string so long that every GPT partition on earth likely has its own unique identifier.

This system doesn’t have MBR’s limits. Drives can be much, much larger and size limits will depend on the operating system and its file systems. GPT allows for a nearly unlimited amount of partitions, and the limit here will be your operating system — Windows allows up to 128 partitions on a GPT drive, and you don’t have to create an extended partition.

On an MBR disk, the partitioning and boot data is stored in one place. If this data is overwritten or corrupted, you’re in trouble. In contrast, GPT stores multiple copies of this data across the disk, so it’s much more robust and can recover if the data is correupted. GPT also stores cyclic redundancy check (CRC) values to check that its data is intact — if the data is corrupted, GPT can notice the problem and attempt to recover the damaged data from another location on the disk. MBR had no way of knowing if its data was corrupted — you’d only see there was a problem when the boot process failed or your drive’s partitions vanished.

source: http://www.howtogeek.com/193669/whats-the-difference-between-gpt-and-mbr-when-partitioning-a-drive/

Now on to the main topic!!!
Now on my setup I used the Ubuntu 15.04 x64 LiveCD and I extracted the ISO to my HDD first partition. It uses Grub2 to boot. Note that I did not remove my Grub4DOS in the MBR and don't worry as UEFI will not use MBR if you configured BIOS to boot UEFI.

Here's the folder where UEFI will search into as said above "/EFI/BOOT/" and files bootx64.efi & grubx64.efi is present in that folder.
On folder /boot/grub/ is where the menu is located and the files it needs. If you want to modify the menu like me find the file "grub.cfg".

I modified it to my liking and here is the new content. You may compare it to the original after you downloaded your own Ubuntu 15.04 ISO.

=====
if loadfont /boot/grub/font.pf2 ; then
set gfxmode=800x600
insmod efi_gop
insmod efi_uga
insmod gfxterm
insmod vbe
insmod jpeg
terminal_output gfxterm
fi

background_image /boot/grub/BootLogo.jpg     < --- I created my own boot logo
set menu_color_normal=brown/black
set menu_color_highlight=black/light-gray

menuentry "Boot Ubuntu GNOME 15.04 X64" {
set gfxpayload=keep
linux /casper/vmlinuz.efi  file=/cdrom/preseed/ubuntu-gnome.seed boot=casper quiet splash ---
initrd /casper/initrd.lz
}

menuentry "Boot Microsoft Windows x64" {   < --- I added this for my Windows boot
    insmod part_gpt
    insmod fat
    insmod search_fs_uuid
    insmod chain
    search --fs-uuid --no-floppy --set=root CHAN-GEME    < --- set the volume serial of the second partition
    chainloader /efi/boot/bootx64.efi
}
=====

My HDD has 3 partitions actually. They are:
1. Boot partition (8GB Fat32) - Named as GrubEFI or "Grub & EFI" as it boots Grub4DOS & Grub2 EFI
2. Scratch parition (8GB Fat32) - This is my second partition where I can change to any OS I want. I just delete and copy the new OS installer here.
3. Data Parition (the rest of the space & NTFS) - as it says just "Data". Also it stores my OS backup for SCRATCH partition.

That's all! I just laid out the files on the primary partition and Ubuntu 15.04 Grub2 works out of the box. The menu entry for Windows is optional. Note that you don't need Grub2 to boot windows, I just want to have a control or a boot menu rather than just it booting directly to only 1 OS. You could copy your Windows x64 installer (note that the EFI files needs to be placed if not yet set for example Win7) starting Win8 and higher EFI is already default set so you can copy it directly to the primary partition.

Restarting network interface in Ubuntu starting 14.04 and up

After Ubuntu 12.04 the /etc/init.d/networking restart command will not work. So it is advised to create a restart script to be able to restart the network interface.

here are the steps done in creating the network restart script.
# vim nrestart.sh
This will create a new file. Then insert the following


ifdown eth0 && ifup eth0

Save after. Note replace the interface to suit your needs.

# chmod a+x nrestart.sh
This will make our script executable

# ./nrestart.sh
This is how to execute the command if you are in the same path.

Squid 3 Basic configuration

This is just an update from my old Squid installation tutorial. Below is some minor changes on the configuration. Tested on Ubuntu 14.04 using Squid 3.3.8


visible_hostname verZionSQD3
hierarchy_stoplist cgi-bin ?

acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
#Recommended minimum configuration:
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl Safe_ports port 5242        # for Viber
acl Safe_ports port 4244        # for Viber
acl Safe_ports port 5243        # for Viber
acl Safe_ports port 9785        # for Viber
acl CONNECT method CONNECT

#http_access deny !Safe_ports
#http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localhost


http_port 3128
hierarchy_stoplist cgi-bin ?
access_log /var/log/squid3/access.log squid

cache_dir ufs /var/spool/squid3 3000 16 256

#We recommend you to use the following two lines.
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
cache deny SSL_ports

#Suggested default:
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern .               0       20%     4320
# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid3


# not to be cached sites place here
acl websitesNC dstdomain www.bir.gov.ph
acl websitesNC dstdomain www.sss.gov.ph
always_direct allow websitesNC

########## Block List ###############
#block sample
acl blocked01 dstdomain www.yahoo.com
acl blocked01 dstdomain www.friendster.com
http_access deny blocked01

###############
# blacklisted #
###############
acl macblock  arp 00:1b:b1:5b:f9:17
http_access deny macblock


##### users with restrictions set #######

#Allow your subnet
acl PChosts src 192.168.1.0/24
http_access allow PChosts

#allow all subnet
#http_access allow all


Enjoy!

Round Robin Internet Failover Setup

Ahoy! It's been a while and been to busy. Here is another of my personal projects and I hope this could help others too. I have updated the instructions below so that it would be easier to follow. I hope :)

In this setup I used gateway switching on the WAN side if a fail on the current router occurred.

Below is a sample graphic on how this setup was made…

 

From the Router side:

- the DHCP was disabled

- set their IP for example in setting like 192.168.0.1/24 for the first router, 192.168.0.2/24 for the second and so on.

For the Server:

- Set it’s WAN IP like 192.168.0.10/24 or  whatever  IP you want as long as the Routers and your server are on the same subnet

- Set the LAN side like 192.168.1/24 or what ever IP you like as long as it is not the same subnet as the WAN side

- setup the Server for the LAN side as DHCP, BIND(optional), SQUID(optional).(kindly see the link)

- the IPTABLES for MASQUERADING for LAN for internet sharing. (kindly see the link)

- I used Ubuntu linux as my server in this setup.

Script1 the route table. You can save this as rtable.sh:

#!/bin/bash

# This is my route add script.

# created by verZion 2/26/2015
# place this in /etc/rc.local for this to run at startup

#clear the current route table
/sbin/route del -net 0.0.0.0
/sbin/route del -net 0.0.0.0
/sbin/route del -net 0.0.0.0

#deploy the route table. Note that the last executed is the primary used gateway
/sbin/route add -net 0.0.0.0 netmask 0.0.0.0 gw 192.168.0.3
/sbin/route add -net 0.0.0.0 netmask 0.0.0.0 gw 192.168.0.2
/sbin/route add -net 0.0.0.0 netmask 0.0.0.0 gw 192.168.0.1

#end of script

- make rtable.sh executable
# chmod a+x rtable.sh
 
- To test Script1 execute it manually and to see if the last executed route is in the beginning of the output list.
# ./rtable.sh 
# netstat -nr            (to check the route table)
Example output on the deployed route table...
$ netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG        0 0          0 eth0
0.0.0.0         192.168.0.2     0.0.0.0         UG        0 0          0 eth0
0.0.0.0         192.168.0.3     0.0.0.0         UG        0 0          0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 eth1
192.168.0.0     0.0.0.0         255.255.255.0   U         0 0          0 eth0

- If there are no problems encountered on the rtable.sh script put it in /etc/rc.local file. Just put it above exit 0 line 
ex: 
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

 /home/verzion/rtable.sh  

 exit 0

#####

Script2 the failover:

- Below is the script I created for the round robin internet failover. Save it as filename netfailover.sh

#!/bin/bash

# This is my internet failover script.

# created by verZion 2/26/2015

# Placed this in cron and set it to run every 5 mins to check the connection.

# Why 5mins? It's because it will download a 2MB file to determine the

# consistency of the connection.

# If you wish for a lighter checking uncomment the PING line below and set it to cron to check

# every 1minute. Note cron smallest interval is 1 minute, no seconds.
# updated 3/6/2015 - found some bugs on the loop statement

#failsw Legend

# 0 = do the check.

# 1 = failure encountered. Email the admins.

# 2 = critical error stop the email until the admin do something to fix the problem

#initialize some variables

loop1=1

dum01=0

failsw=0
gwh=3

#uncomment dURL if you are going to use download as check tool

#dURL="http://squid.acmeconsulting.it/download/squid-2.7.STABLE8-bin.zip"

#get the current route table and put it in dump01 file

/bin/netstat -nr > /tmp/dump01

#Set the maximum gateways in the network

maxcon=3

#Start testing and changing the connection on failure

while [  $loop1 -le $maxcon ] && [ $failsw -ne 2 ]; do

        gw=$(awk '{print $2};' /tmp/dump01 | head -$gwh | tail -1 )

         echo Checking connection on $gw... >> /var/log/syslog

       #download the file

        #wget --limit-rate=30k --tries=1 --connect-timeout=10 --read-timeout=5  --no-check-certificate --output-document=/tmp/dummy $dURL &> /dev/null

        #if you wish to use ping instead of download uncomment the line below

        #and comment the wget line above

        ping zdnet.com -c3 &> /dev/null

        #check the return value where non 0 means we have a problem

        if [[ "$?" -ne 0 ]]; then

            let loop1=$loop1+1

            failsw=1

            if [[ $loop1 -le $maxcon ]]; then  #if less than or equal change the route table

                 echo network problem detected!!! Changing gateway...

                 echo clearing the route configuration...

                 let loop2=$maxcon-1

                 while [  $loop2 -ne 0 ]; do

                        echo delete route $loop2

                        /sbin/route del -net 0.0.0.0

                        let loop2=$loop2-1

                 done

                 echo then shuffling the route table...        

                 loop2=3

                 let dum01=$maxcon+2

                 let loop4=$loop1+2

                 while [ $loop2 -lt $dum01 ] || [ $loop2 -eq $dum01 ]; do

                        if [[ $loop2 -ne $loop4 ]]; then

                            gw=$(awk '{print $2};' /tmp/dump01 | head -$loop2 | tail -1 )

                            echo loading gateway $gw

                            /sbin/route add -net 0.0.0.0 netmask 0.0.0.0 gw $gw

                       fi

                       let loop2=$loop2+1

                 done

                 #the last route is the primary gateway

                 let dum01=$loop1+2

                 gw=$(awk '{print $2};' /tmp/dump01 | head -$dum01 | tail -1 )

                 echo then loading default gateway $gw

                 /sbin/route add -net 0.0.0.0 netmask 0.0.0.0 gw $gw

           fi
           let gwh=$gwh+1

        else

            echo no network problem detected. Exiting!

            break     #exit loop

        fi

done

#check if all gateways failed

if  [[ $loop1 -gt $maxcon ]]; then

    echo Warning all gateways failed!!!

    sed -i '20s/failsw=0/failsw=2/' /home/verzion/netfailover.sh   #This means to stop the checking until failsw is set to 0

    #email the admin about it

    /usr/bin/mail -s "Server reporing: MAJOR Problem on the internet connection!!!" verzion@internal.com <<< "All gateways are down! :( After fixing the problem reset the failsw to 0 to enable the autocheck again."

fi

if [[ $failsw -eq 1 ]]; then

    #email the admin about the current gateway changes

    echo Warning 1 of the internet gateway has failed. Currently gateway $gw is loaded.

    /usr/bin/mail -s "Server reporing: Internet failure detected!" verzion@internal.com <<< "The internet gateway was changed to $gw"

fi

if [[ $failsw -eq 2 ]]; then

    echo Major network downtime detected. Reset the failsw variable to 0 after fixing the problem

fi

#end of script

- set the permission on the script files so it will become executable.

# chmod a+x netfailover.sh

- you may test it manually by executing it like example below

# ./netfailover.sh

- If no problems occurred on the manual execution now edit /etc/crontab and add this below the lines.

* * * * *   root    sleep 10; /home/verzion/netfailover.sh >> /var/log/syslog

- No need to restart cron as it will see the changes after the save.

- to see the log files execute the command below…

# tail -f /var/log/syslog

- Below is a sample log output from syslog…

Mar  2 12:17:01 VZServer CRON[32463]: (root) CMD (   sleep 10; /home/verzion/netfailover.sh >> /var/log/syslog)

Checking connection on 192.168.0.1...

no network problem detected. Exiting!


Summary on the Setup

- Note on the script there is a line that mails the admin as notification. I have setup an internal mail and the setup for the internal mail is not covered here.

- Also on the Script2 the BLUE colored text is the one that you can modify according to your needs and the VIOLET colored is the one you need to reset if a total failure was met in the script.

- Below are some details on the colored BLUE on Script2 (netfailover.sh)

#dURL="http://squid.acmeconsulting.it/download/squid-2.7.STABLE8-bin.zip"
If you are going to use the download test just remove the remark(#) and put the remark on ping line. In short use only either of the two.

#Set the maximum gateways in the network

maxcon=3

You set the number of routers here and the lowest is 2. Please do not use this script for only 1 router. It would be senseless.

ping zdnet.com -c3
Here you may use any URL you wish. You can ping facebook.com if you want.
on the -c3 option of the ping command this says how many times the ping should run where it is set to 3. You may set it higher or lower according to your needs.

/home/verzion/netfailover.sh
This states where the netfailover.sh script was placed on your computer. Do not forget to change this as the path is needed to be correct.

"Server reporing: MAJOR Problem on the internet connection!!!" verzion@internal.com
All under the mailing line this sends mail to the internal mail I set up on my server. The notes for the mail server setup is not found on this documentation. Kindly search for it as there are tons of working tutorials out there.
If you do not wish to use this as of now you may remark(#) the mail command line.


Have fun coding!!!