Tuesday, March 31, 2015

Squid 3 Basic configuration

This is just an update from my old Squid installation tutorial. Below is some minor changes on the configuration. Tested on Ubuntu 14.04 using Squid 3.3.8


visible_hostname verZionSQD3
hierarchy_stoplist cgi-bin ?

acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
#Recommended minimum configuration:
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl Safe_ports port 5242        # for Viber
acl Safe_ports port 4244        # for Viber
acl Safe_ports port 5243        # for Viber
acl Safe_ports port 9785        # for Viber
acl CONNECT method CONNECT

#http_access deny !Safe_ports
#http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localhost


http_port 3128
hierarchy_stoplist cgi-bin ?
access_log /var/log/squid3/access.log squid

cache_dir ufs /var/spool/squid3 3000 16 256

#We recommend you to use the following two lines.
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
cache deny SSL_ports

#Suggested default:
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern .               0       20%     4320
# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid3


# not to be cached sites place here
acl websitesNC dstdomain www.bir.gov.ph
acl websitesNC dstdomain www.sss.gov.ph
always_direct allow websitesNC

########## Block List ###############
#block sample
acl blocked01 dstdomain www.yahoo.com
acl blocked01 dstdomain www.friendster.com
http_access deny blocked01

###############
# blacklisted #
###############
acl macblock  arp 00:1b:b1:5b:f9:17
http_access deny macblock


##### users with restrictions set #######

#Allow your subnet
acl PChosts src 192.168.1.0/24
http_access allow PChosts

#allow all subnet
#http_access allow all



Enjoy!

Tuesday, March 3, 2015

Round Robin Internet Failover Setup



Ahoy! It's been a while and been to busy. Here is another of my personal projects and I hope this could help others too. I have updated the instructions below so that it would be easier to follow. I hope :)

In this setup I used gateway switching on the WAN side if a fail on the current router occurred.
Below is a sample graphic on how this setup was made…


 





From the Router side:
- the DHCP was disabled
- set their IP for example in setting like 192.168.0.1/24 for the first router, 192.168.0.2/24 for the second and so on.

For the Server:
- Set it’s WAN IP like 192.168.0.10/24 or  whatever  IP you want as long as the Routers and your server are on the same subnet
- Set the LAN side like 192.168.1/24 or what ever IP you like as long as it is not the same subnet as the WAN side
- setup the Server for the LAN side as DHCP, BIND(optional), SQUID(optional).(kindly see the link)
- the IPTABLES for MASQUERADING for LAN for internet sharing. (kindly see the link)
- I used Ubuntu linux as my server in this setup.

Script1 the route table. You can save this as rtable.sh:
#!/bin/bash
# This is my route add script.
# created by verZion 2/26/2015
# place this in /etc/rc.local for this to run at startup

#clear the current route table
/sbin/route del -net 0.0.0.0
/sbin/route del -net 0.0.0.0
/sbin/route del -net 0.0.0.0

#deploy the route table. Note that the last executed is the primary used gateway
/sbin/route add -net 0.0.0.0 netmask 0.0.0.0 gw 192.168.0.3
/sbin/route add -net 0.0.0.0 netmask 0.0.0.0 gw 192.168.0.2
/sbin/route add -net 0.0.0.0 netmask 0.0.0.0 gw 192.168.0.1
#end of script

- make rtable.sh executable
# chmod a+x rtable.sh
 
- To test Script1 execute it manually and to see if the last executed route is in the beginning of the output list.
# ./rtable.sh 
# netstat -nr            (to check the route table)
Example output on the deployed route table...
$ netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG        0 0          0 eth0
0.0.0.0         192.168.0.2     0.0.0.0         UG        0 0          0 eth0
0.0.0.0         192.168.0.3     0.0.0.0         UG        0 0          0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 eth1
192.168.0.0     0.0.0.0         255.255.255.0   U         0 0          0 eth0


- If there are no problems encountered on the rtable.sh script put it in /etc/rc.local file. Just put it above exit 0 line 
ex: 
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.


 /home/verzion/rtable.sh  


 exit 0

#####
Script2 the failover:
- Below is the script I created for the round robin internet failover. Save it as filename netfailover.sh
#!/bin/bash
# This is my internet failover script.
# created by verZion 2/26/2015
# Placed this in cron and set it to run every 5 mins to check the connection.
# Why 5mins? It's because it will download a 2MB file to determine the
# consistency of the connection.
# If you wish for a lighter checking uncomment the PING line below and set it to cron to check
# every 1minute. Note cron smallest interval is 1 minute, no seconds.
# updated 3/6/2015 - found some bugs on the loop statement

#failsw Legend
# 0 = do the check.
# 1 = failure encountered. Email the admins.
# 2 = critical error stop the email until the admin do something to fix the problem


#initialize some variables
loop1=1
dum01=0
failsw=0
gwh=3
#uncomment dURL if you are going to use download as check tool
#dURL="http://squid.acmeconsulting.it/download/squid-2.7.STABLE8-bin.zip"

#get the current route table and put it in dump01 file
/bin/netstat -nr > /tmp/dump01

#Set the maximum gateways in the network
maxcon=3

#Start testing and changing the connection on failure
while [  $loop1 -le $maxcon ] && [ $failsw -ne 2 ]; do

        gw=$(awk '{print $2};' /tmp/dump01 | head -$gwh | tail -1 )
         echo Checking connection on $gw... >> /var/log/syslog
       #download the file
        #wget --limit-rate=30k --tries=1 --connect-timeout=10 --read-timeout=5  --no-check-certificate --output-document=/tmp/dummy $dURL &> /dev/null

        #if you wish to use ping instead of download uncomment the line below
        #and comment the wget line above
        ping zdnet.com -c3 &> /dev/null

        #check the return value where non 0 means we have a problem
        if [[ "$?" -ne 0 ]]; then
            let loop1=$loop1+1
            failsw=1

            if [[ $loop1 -le $maxcon ]]; then  #if less than or equal change the route table
                 echo network problem detected!!! Changing gateway...
                 echo clearing the route configuration...
                 let loop2=$maxcon-1
                 while [  $loop2 -ne 0 ]; do
                        echo delete route $loop2
                        /sbin/route del -net 0.0.0.0
                        let loop2=$loop2-1
                 done

                 echo then shuffling the route table...        
                 loop2=3
                 let dum01=$maxcon+2
                 let loop4=$loop1+2
                 while [ $loop2 -lt $dum01 ] || [ $loop2 -eq $dum01 ]; do
                        if [[ $loop2 -ne $loop4 ]]; then
                            gw=$(awk '{print $2};' /tmp/dump01 | head -$loop2 | tail -1 )
                            echo loading gateway $gw
                            /sbin/route add -net 0.0.0.0 netmask 0.0.0.0 gw $gw
                       fi
                       let loop2=$loop2+1
                 done
                 #the last route is the primary gateway
                 let dum01=$loop1+2
                 gw=$(awk '{print $2};' /tmp/dump01 | head -$dum01 | tail -1 )
                 echo then loading default gateway $gw
                 /sbin/route add -net 0.0.0.0 netmask 0.0.0.0 gw $gw
           fi
           let gwh=$gwh+1
        else
            echo no network problem detected. Exiting!
            break     #exit loop
        fi
done

#check if all gateways failed
if  [[ $loop1 -gt $maxcon ]]; then
    echo Warning all gateways failed!!!
    sed -i '20s/failsw=0/failsw=2/' /home/verzion/netfailover.sh   #This means to stop the checking until failsw is set to 0
    #email the admin about it
    /usr/bin/mail -s "Server reporing: MAJOR Problem on the internet connection!!!" verzion@internal.com <<< "All gateways are down! :( After fixing the problem reset the failsw to 0 to enable the autocheck again."
fi
if [[ $failsw -eq 1 ]]; then
    #email the admin about the current gateway changes
    echo Warning 1 of the internet gateway has failed. Currently gateway $gw is loaded.
    /usr/bin/mail -s "Server reporing: Internet failure detected!" verzion@internal.com <<< "The internet gateway was changed to $gw"
fi
if [[ $failsw -eq 2 ]]; then
    echo Major network downtime detected. Reset the failsw variable to 0 after fixing the problem
fi

#end of script

- set the permission on the script files so it will become executable.
# chmod a+x netfailover.sh

- you may test it manually by executing it like example below
# ./netfailover.sh

- If no problems occurred on the manual execution now edit /etc/crontab and add this below the lines.
* * * * *   root    sleep 10; /home/verzion/netfailover.sh >> /var/log/syslog

- No need to restart cron as it will see the changes after the save.

- to see the log files execute the command below…
# tail -f /var/log/syslog

- Below is a sample log output from syslog…
Mar  2 12:17:01 VZServer CRON[32463]: (root) CMD (   sleep 10; /home/verzion/netfailover.sh >> /var/log/syslog)
Checking connection on 192.168.0.1...
no network problem detected. Exiting!


Summary on the Setup
- Note on the script there is a line that mails the admin as notification. I have setup an internal mail and the setup for the internal mail is not covered here.

- Also on the Script2 the BLUE colored text is the one that you can modify according to your needs and the VIOLET colored is the one you need to reset if a total failure was met in the script.

- Below are some details on the colored BLUE on Script2 (netfailover.sh)

#dURL="http://squid.acmeconsulting.it/download/squid-2.7.STABLE8-bin.zip"
If you are going to use the download test just remove the remark(#) and put the remark on ping line. In short use only either of the two.

#Set the maximum gateways in the network
maxcon=3
You set the number of routers here and the lowest is 2. Please do not use this script for only 1 router. It would be senseless.

ping zdnet.com -c3
Here you may use any URL you wish. You can ping facebook.com if you want.
on the -c3 option of the ping command this says how many times the ping should run where it is set to 3. You may set it higher or lower according to your needs.

/home/verzion/netfailover.sh
This states where the netfailover.sh script was placed on your computer. Do not forget to change this as the path is needed to be correct.

"Server reporing: MAJOR Problem on the internet connection!!!" verzion@internal.com
All under the mailing line this sends mail to the internal mail I set up on my server. The notes for the mail server setup is not found on this documentation. Kindly search for it as there are tons of working tutorials out there.
If you do not wish to use this as of now you may remark(#) the mail command line.


Have fun coding!!!