How to create automated backups for your web server?

If you have a web server running one or many websites or applications and hosting a database server it may be a good idea to have an automated backup process.

Most (or all) Web Hosting providers will charge you for such service, but the good news is that you can create your own automated backup process. Do not wait until you wake up a morning and discover that your homepage has changed into this :

You have boon hacked

The following tutorial has been written and tested with the following configuration:

  • Operating System : Ubuntu Linux 14.04.4 with root access
  • Web Server : Apache2
  • Database Server : Mysql 5.5.49
  • Scripting tool : Unix Shell

You can adapt the script or paths to meet your configuration’s needs. I will not cover any  external file transfer process, the files will be stored on the same server, but it’s not a big deal to automate the transfer of files to an other server.

Before we start, I will summarize the backup strategy .

  1. Create folder structures that will contain the backup files;
  2. The backup program consists of a shell script (.sh) that runs everyday at a fixed hour using a CRON task. The logic of that script will be the following :
    • Rename the existing (yesterday’s files) backup files by prefixing them with the the current date;
    • Delete backups older than X days (in our case it will be 5 days);
    • Take backupa of databases and compress the files;
    • Take backup of the Apache www folder and compress it.
  3. And finally, create the CRON task and set the appropriate permissions.

 

Folder Structure

We will create two folders:

  • /bkp : will be used to store the backup files ;
  • /opt/backup/ : will contain the backup script (backup.sh) .

To create the two folder execute the following commands :

mkdir /bkp
mkdir /opt/backup

 

Backup Script

We start by creating the script and give it the right permissions. On the command line execute the following commands:

# create the script file
touch /opt/backup/backup.sh

# make the file executable
chmod +x /opt/backup/backup.sh

Then we will start editing the script using VI editor (or your preferred text editor):

vi /opt/backup/backup.sh

Since it’s a shell script, it must start with the following line :

#!/bin/sh

The following sections will cover content that will be added to the backup.sh shell script.

Rename the old backup files

The next steps of the script will consist of creating new backup files. We don’t want our backup files to be replaced by the new ones. We will prefix the last created backup files with the current day’s date.

Add these lines to the script:

# rename existing backup files

## Switch to the backups folder
cd /bkp

## get the current date
current_date=$(date +"%Y-%m-%d")
shopt -s extglob

## Rename the files not starting with a date
for file in !([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]*.gz); do
  mv "$file" "${current_date}_$file"
done

 

Delete backups older than X days

Your server space may be limited. We have to make a decision about the number of days we want to keep. For this tutorial we will choose 5 days.

# delete backups older than 5 days under /bkp folder
find /bkp -type f -name '*' -mtime +5 -exec rm {} \;

 

Take database backups

We will use the mysqldump command to create backups of the databases. Lets say we have 3 mysql databases : db1, db2 and db3. The backups will consist of what we call mysql dumps .

In addition, we will compress these mysql dumps (or backup files) in order to save space on the server.

Add the following lines to the script (replace database names, usernames and passwords with you own values):

# take compressed backups of databases
mysqldump -u root yourPassword db1| gzip -9 > /bkp/db1.sql.gz
mysqldump -u root yourPassword db2| gzip -9 > /bkp/db2.sql.gz
mysqldump -u root yourPassword db3| gzip -9 > /bkp/db3.sql.gz

 

Take folders backup

Now we will add the commands to backup our website’s files and folders. For Apache web server, website files or commonly stored in /var/www, but it’s not a rule, configuration my change. For this tutorial we suppose that our files are located in /var/www folder .

To take a backup, we will simply use the tar command to create a compressed archive for the whole www folder.

Add the following lines to the script (Replace the paths with you own values) :

# take compressed backup of important folders
tar -zcf /bkp/www.tar.gz /var/www/

Note: you can also use the same command to take a backup of any important folders located on your server.

Save the backup.sh file. Here is the final structure of the script :

#!/bin/sh

# rename existing backup files

# Switch to the backups folder
cd /bkp

# get the current date
current_date=$(date +"%Y-%m-%d")
shopt -s extglob

# Rename the files not starting with a date
for file in !([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]*.gz); do
  mv "$file" "${current_date}_$file"
done

# delete backups older than 5 days under /bkp folder
find /bkp -type f -name '*' -mtime +5 -exec rm {} \;

# take compressed backups of databases
mysqldump -u root yourPassword db1| gzip -9 > /bkp/db1.sql.gz
mysqldump -u root yourPassword db2| gzip -9 > /bkp/db2.sql.gz
mysqldump -u root yourPassword db3| gzip -9 > /bkp/db3.sql.gz

# take compressed backup of important folders
tar -zcf /bkp/www.tar.gz /var/www/

 

Create the CRON task

Now that our script is ready, you can test the process by executing it and make sure it has the behavior you wanted. When satisfied with the results, you can automate the process using the CRON tasks.

For this tutotrial, we want our script be executed every day at 00:01 AM. On a shell command line execute the following command to start editing the crontab :

sudo crontab -e

Then add the following line and save the file :

01 0 * * * /opt/backup/backup.sh

At this point everything must be OK, you have to wait until 00:01 AM to see the results.

Restoring the files

If you need to use the saved backup files simply do the following:

MySQL dumps

  1. Uncompress the files usinng gunzip command ( ex: gunzip db1.gz)
  2. This will produce an SQL file.
  3. Import the SQL file using any importing tool (phpmyadmin, MySQL Workbench, MySQL Administrator, etc)
  4. Sometimes you will experience some problems with file encoding, in most cases you have to remove or change some parameters.

tar archives

  1. Just uncompress the tar archive and copy the files in their appropriate location which is /var/ww for this tutorial. (Example : tar xzvf www.tar.gz)

 

 

How to create directories in Java

In java you can create a directory (or directories) using the standard IO package or using the java NIO package.

Create directories using the java IO package

// create a single directory
new File("c:\\myDir").mkdir();

// create a directory and all its sub-directories
new File("c:\\myRootDir\\Subdir1\\SubDir2").mkdirs();

 

Create directories using the java NIO package

This will work only on Java 7 and above.

Path path = Paths.get("C:\\Mydir");
Files.createDirectories(path);

You have to import the classes java.nio.file.Paths and java.nio.file.Files .

 

 

 

How to access to class private members in Java

It is possible to use to access class private instance variables and methods using Reflexion.

Access to private instance variables

MyClass myClass = new MyClass();
Field field = MyClass.class.getDeclaredField("myField");
field.setAccessible(true);
Object value = field.get(myClass);
System.out.println(value);

 

Access to private methods

MyClass myClass = new MyClass();
Method method = myClass.getClass().getDeclaredMethod("myMethodName");
method.setAccessible(true);
Object o = method.invoke(object);

 

How to connect to a database using JDBC

This example shows how to connect to a database using JDBC. (In this example we will connect to Oracle database) .

You will need to add the odbc{x}.jar library to your classpath, downloadable from Oracle’s website.

import java.sql.DriverManager;
import java.sql.Connection;
import java.sql.SQLException;

public class JdbcExample {

 public static void main(String[] args) {
  
  try{
  	// register the driver
  	Class.forName("oracle.jdbc.driver.OracleDriver");
  	// create the connexion
  	Connection connection
            =DriverManager.getConnection(
               "jdbc:oracle:thin:@localhost:1521:xe", "system", "pwd"
                );
  } catch (ClassNotFoundException e) {
    // the driver was not found
    e.printStackTrace();
  } catch (SQLException e) {
    // Error when connecting to database
    e.printStackTrace();
  }
 }

}

 

How to disable SSL certificat validation in Java

This fix will disable the SSL certificate validation. It is not recommanded in a production environment. A recommanded approach is to install the needed certificates on the JVM.

Create the following class:

import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;


public class SSLFix {
 
 public static void execute(){
  TrustManager[] trustAllCerts = new TrustManager[] {
        new X509TrustManager() {
          public java.security.cert.X509Certificate[] getAcceptedIssuers() {
           return null;
          }
          @Override
          public void checkClientTrusted(X509Certificate[] arg0, String arg1)
           throws CertificateException {}
 
          @Override
          public void checkServerTrusted(X509Certificate[] arg0, String arg1)
            throws CertificateException {}

          }
     };

  SSLContext sc=null;
  try {
   sc = SSLContext.getInstance("SSL");
  } catch (NoSuchAlgorithmException e) {
   e.printStackTrace();
  }
  try {
   sc.init(null, trustAllCerts, new java.security.SecureRandom());
  } catch (KeyManagementException e) {
   e.printStackTrace();
  }
  HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());

  // Create all-trusting host name verifier
  HostnameVerifier validHosts = new HostnameVerifier() {
  @Override
  public boolean verify(String arg0, SSLSession arg1) {
   return true;
  }
  };
  // All hosts will be valid
  HttpsURLConnection.setDefaultHostnameVerifier(validHosts);
 }

}

 

Now whenever you want to create a SSL connection, add the following line just before the connection:

SSLFix.execute();

 

How to read and write Properties file in java 8

In Java, properties files are used to store data and configuration. This tutorial will show you how to store and read properties files in Java 8.

Using InputStream

Write to properties file

import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.util.Properties;

public class PropertiesFileSample {
 
 final private static String PROPERTIES_FILE_PATH="config.properties";

 public static void main(String[] args) {
  
  Properties properties = new Properties();
  
  // using java 8 try-with-resources structure,
  // so the output stream will be closed automatically
  try(OutputStream outputStream = new FileOutputStream(PROPERTIES_FILE_PATH)){
   
   // set some values
   properties.setProperty("username", "sampleusername");
   properties.setProperty("url", "tutoref.com");
   
   // store the values
   properties.store(outputStream, null);
   
  } catch (IOException e) {
   e.printStackTrace();
  } 
 }

}

The result will look like this :

#Sat Jul 08 23:55:53 EDT 2017
url=tutoref.com
username=sampleusername

 

Read from a properties file

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;

public class PropertiesFileSample {
 
 final private static String PROPERTIES_FILE_PATH="config.properties";

 public static void main(String[] args) {
 
  Properties properties = new Properties();
 
  // using java 8 try-with-resources structure
  // so the input stream will be closed automatically
  try(InputStream inputStream = new FileInputStream(PROPERTIES_FILE_PATH)){
 
  // read a value
  String url = properties.getProperty("url");
 
  // display the value
  System.out.println(url);
 
 } catch (IOException e) {
  e.printStackTrace();
 } 
 }

}

 

Load properties from classpath

import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;

public class PropertiesFileSample {
 
 final private static String PROPERTIES_FILE_PATH="config.properties";

 public static void main(String[] args) {
  
  Properties properties = new Properties();
  
  // using java 8 try-with-resources structure
  // so the input stream will be closed automatically
  try(
    InputStream inputStream=
    PropertiesFileSample.class.getResourceAsStream(PROPERTIES_FILE_PATH)
    ){
   
   // load the properties
   properties.load(inputStream);
   
   // read a value
   String url = properties.getProperty("url");
   
   // display the value
   System.out.println(url);
   
  } catch (IOException e) {
   e.printStackTrace();
  } 
 }

}

 

If you want to load the properties file from a non-static method, you should use the following:

InputStream is = getClass().getClassLoader().getResourceAsStream(PROPERTIES_FILE_PATH);

 

 

How to connect to an HTTPS page in java using HttpsURLConnection

The folllowing code will connect to a https page (using SSL), get the response code and content.

 

If you have timeout or networking issues, check if you don’t have to modify the firewall configuration or install a SSL certificate on your JVM.

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import javax.net.ssl.HttpsURLConnection;

public class SampleHttpsClient {
 
 public static void main(String[] args) {
  
  String host = "https://www.google.com";
  try {
   
   URL url = new URL(host);
   HttpsURLConnection httpsURLConnection = (HttpsURLConnection) url.openConnection();
   
   // get response code, 200 = Success
   int responseCode = httpsURLConnection.getResponseCode();
   System.out.println(responseCode);
   
   // if response == 200, display the content
   if(responseCode==200){
    System.out.println(httpsURLConnection.getContent().toString());
    BufferedReader br = new BufferedReader(new InputStreamReader(httpsURLConnection.getInputStream()));
    String line=null;
    while((line = br.readLine())!= null){
     System.out.println(line);
    }
   }
   
  } catch (IOException e) {
   e.printStackTrace();
  } 
  
 }
 
}

 

How to send url to Intent in Android

This code will open the link in your Android browser. We suppose you have an Activity and you did the correct imports :

 

String url = "http://www.tutoref.com";
Intent intent = new Intent(Intent.ACTION_VIEW);
intent.setData(Uri.parse(url));
startActivity(intent);

 

The necessary imports are:

import android.content.Intent;
import android.net.Uri;
import android.os.Bundle;

 

How to convert Date into LocalDateTime in Java 8

Date to LocalDateTime

Date date = new Date();
Instant instant = date.toInstant();
LocalDateTime ldt = LocalDateTime.ofInstant(instant, ZoneId.systemDefault());

 

LocalDateTime to Date

LocalDateTime ldt = LocalDateTime.now();
ZonedDateTime zdt = ldt.atZone(ZoneId.systemDefault());
Date date = Date.from(zdt.toInstant());