Sync calendars and address books between the Linux console and Android

Calendars and address books contain very sensible personal data. So it’s not advisable to share them with companies like Google or Apple. But with more than one client system in use, it is necessary to run an own calendar server to synchronize that data between the clients.

The first part of this article describes the installation and configuration of a calendar and contacts server. After that follows the presentation of calendar and address book programs for the Linux console and the Android operating system.

Table of contents

• 1. Calendar and contacts server
  • 1.1 Baikal
  • 1.2 DynDNS
• 2. Linux console
  • 2.1 Prerequisites
  • 2.2 Vdirsyncer
  • 2.3 Khard
  • 2.4 Khal
  • 2.5 Import calendars
• 3. Android

Calendar and contacts server

There are quite a number of calendar and contacts servers, which implement the CalDAV and CardDAV protocol. Among others you can choose between OwnCloud, Baïkal and Darwin Calendar Server. I used the last one for a few years but switched to Baikal recently. It’s relatively easy to setup and it comes with a web interface to create calendars and address books.

Baikal

You need a web server to run Baikal. I use LightTPD. You can find installation instructions here. For this tutorial I assume, that the web server is configured to accept encrypted connections. This article provides information about the creation of a valid SSL certificate.

After the web server is running, install SQLite:

sudo aptitude install sqlite php5-sqlite

Then download the latest release from the Baikal website and extract it into the content folder of the web server:

cd /tmp
wget http://baikal-server.com/get/baikal-flat-0.2.7.zip
unzip baikal-flat-0.2.7.zip
sudo mv baikal-flat /var/www/baikal
sudo chown -R www-data:www-data /var/www/baikal

Before you can start the setup process, you have to create the following file:

sudo touch /var/www/baikal/Specific/ENABLE_INSTALL
sudo chown www-data:www-data /var/www/baikal/Specific/ENABLE_INSTALL

Then visit https://example.org/baikal/admin/ and follow the setup instructions. After the setup is complete, log in as Admin and create a new user and as many address books and calendars as you want.

DynDNS

If the server runs at your home, you should register a DynDNS domain. So you can access it from outside too. To achieve that, I use the free service from no-ip.com.

If you can’t run an own server, then choose a more privacy-friendly provider, for example: Fastmail or Posteo.

Linux console

This section contains the instructions for the Linux console. I’ve used Debian but the applications should run under other Linux distributions too. First come some information about system prerequisites. The second subsection describes the synchronization of the address books and calendars between the server and the local machine. Afterwards follow the presentation of the address book and calendar applications. All programs are intended for the console. Therefore no graphical user interface is required.

Prerequisites

Before we can start, we have to fulfill the following requirements:

sudo aptitude install git python-setuptools
sudo easy_install pip
sudo pip install virtualenv

Pip is a package manager for python. For its installation the Setuptools program is required and must be installed first. Virtualenv is used to create a virtual Python environment for every installed program. So it’s possible to use the same Python module in different versions without conflicts. Furthermore it helps to keep your local Python installation clean. To uninstall a previously installed program, you only have to delete it’s virtual environment folder.

First we need a directory for the virtual environments:

mkdir ~/.virtualenvs

Then create a sub folder for the binaries:

mkdir ~/.virtualenvs/bin

Finally add the created binary folder to the shell’s path variable. Here it’s shown for the ZSH:

vim ~/.zshrc
[...]
# add ~/.virtualenvs/bin to the path variable
if [ -d "$HOME/.virtualenvs/bin" ] ; then
    PATH="$HOME/.virtualenvs/bin:$PATH"
fi
[...]

In the following sections we create a virtual environment for every installed program and set a symbolic link from the program binary to the created bin folder. If you want to get additional information about virtualenv, you can find them for example here.

Vdirsyncer

Vdirsyncer is used to synchronize calendar and contact data between the server and the local machine. Install it with the following commands:

# change to the virtualenv folder
cd ~/.virtualenvs
# create a new virtual environment with the name vdirsyncer
virtualenv vdirsyncer
# install vdirsyncer with the local pip version
./vdirsyncer/bin/pip install vdirsyncer
# create symlink
ln -s ~/.virtualenvs/vdirsyncer/bin/vdirsyncer ~/.virtualenvs/bin

Then download the example configuration file:

mkdir ~/.config/vdirsyncer/
wget -O ~/.config/vdirsyncer/config https://github.com/untitaker/vdirsyncer/raw/master/example.cfg

And configure the program:

vim ~/.config/vdirsyncer/config

The config file contents should look like this:

# An example configuration for vdirsyncer.

[general]
# A folder where vdirsyncer can store some metadata about each pair.
status_path = ~/.config/vdirsyncer/status/

# CARDDAV
[pair contacts]
a = contacts_local
b = contacts_remote
collections = ["family", "friends"]
conflict_resolution = a wins

[storage contacts_local]
type = filesystem
path = ~/.contacts/
fileext = .vcf

[storage contacts_remote]
type = carddav
url = https://example.org/baikal/card.php/addressbooks/YOURUSERNAME/
auth = digest
verify = /etc/ssl/local/example.org.pem
username = YOURUSERNAME
password = YOURPASSWORD

# CALDAV
[pair calendar]
a = calendar_local
b = calendar_remote
collections = ["calendar", "holidays", "birthdays"]
conflict_resolution = a wins

[storage calendar_local]
type = filesystem
path = ~/.calendars/
fileext = .ics

[storage calendar_remote]
type = caldav
url = https://example.org/baikal/cal.php/calendars/YOURUSERNAME/
auth = digest
verify = /etc/ssl/local/example.org.pem
username = YOURUSERNAME
password = YOURPASSWORD

The CalDAV section of the vdirsyncer config contains the default calendar and two additional calendars (holidays and birthdays). The CardDAV section contains two address books. Change that to your needs. Furthermore you have to replace the host name and user name in the URL’s. A different calendar server could also require a slightly changed URL schema.

If you use a self signed SSL certificate to secure the connection to your calendar server, you have to import it’s public key into the client system. Therefore create the folder /etc/ssl/local at your Linux client and copy the public key file into that directory.

After configuration, run the synchronization process by typing:

vdirsyncer sync

Lastly create a cron job to sync your contacts and calendars every hour:

crontab -e
[...]
0   *   *   *   * $HOME/.virtualenvs/bin/vdirsyncer sync

You can find further information about the program at vdirsyncer’s project page.

Khard

Khard is an address book client for the Linux console. It lists, creates, modifies and removes vCard-formated contacts data at your local machine. Furthermore it serves as an external address book for the mail clients Mutt and Alot and the SIP client twinkle.

Installation of khard:

cd ~/.virtualenvs
virtualenv khard
./khard/bin/pip install khard
ln -s ~/.virtualenvs/khard/bin/khard ~/.virtualenvs/bin

Next follows the download and modification of khard’s example configuration file:

mkdir ~/.config/khard/
wget -O ~/.config/khard/khard.conf https://github.com/scheibler/khard/raw/master/khard.conf.example

The “addressbooks” section contains the paths to all address books and calendars, which are specified in the vdirsyncer configuration. Furthermore you must set the path to your favorite text editor. It’s required to create new contacts and modify existing ones.

After you are finished you can list all existing contacts with the following command:

khard list

Without any parameters, the contacts of all address books are listed. You can filter only certain address books like this:

khard list -a family,friends

You also can search for someone:

khard list -s max

The overview table only shows the first phone number and email address. To get all contact’s details, type:

khard details

The next command creates a new contact:

khard new -a friends

The form for the new contact opens in your favorite text editor. After you have filled the fields and saved the document, the new contact is created. To cancel, leave the first name, last name and organization fields blank. If you want to modify the created contact later, type:

khard modify

Or remove it with:

khard remove

After you’re finished, you must run the synchronization process:

vdirsyncer sync

It’s possible to use khard as an external address book for Mutt and Twinkle too. For further information about that, please have a look into khard’s Readme file.

Khal

Khal is a calendar application for the Linux console. It lists, creates, modifies and removes CalDAV-formated calendar entries at your local machine.

Installation:

cd ~/.virtualenvs
virtualenv khal
./khal/bin/pip install khal
ln -s ~/.virtualenvs/khal/bin/khal ~/.virtualenvs/bin

Next copy and adapt its example configuration file (should be similar to the khard config):

mkdir ~/.config/khal
wget -O ~/.config/khal/khal.conf https://github.com/geier/khal/raw/master/khal.conf.sample

After the successful configuration you can list all events for the next week like this:

khal agenda --days=7

Or only show appointments of one calendar:

khal agenda --days=7 -a holidays

Create a new single event at November 9th, which lasts from 14:00 till 16:00:

khal new -a calendar -l "optional location" 09.11. 14:00 16:00 Title of appointment

Or create an all-day appointment, which recurs every year:

khal new -a birthdays -r yearly 19.02. Max birthday

Further information about khal can be found here.

Import calendars

At some web sites you can subscribe to CalDAV formated calendars or download them. For example you can add these subscriptions to your Android phone. But then, they aren’t synchronized with your own calendar server and so they are not available at your Linux machine. This section describes, how to download a whole calendar and add it to your own calendar server.

At www.ifeiertage.de you can download a calendar with all German holidays. After downloading you get a single text file with all appointments. But the calendar server only supports .ics files with a single one inside. Therefore you have to split the text file into multiple appointment files. This can be done with the python script split_vcalendar.py:

cd ~/.local/bin
wget http://eric-scheibler.de/downloads/split_vcalendar.py
chmod +x split_vcalendar.py

Create a new calendar at your calendar server and enter it into your vdirsyncer config file. Then download and split the calendar file and move the .ics files into the calendar folder. In this example I’ve downloaded and processed the holidays for the German state Saxony:

cd /tmp
wget -O feiertage.ics http://www.ifeiertage.de/calendar.php\?bl\=sn\&o\=o1\&o2\=sonder\&o4\=zeitum\&t\=dnl
split_vcalendar.py feiertage.ics feiertage
mv feiertage/*.ics ~/.calendars/feiertage/
vdirsyncer sync

Android

Android has no build in support for CalDAV and CardDAV. Therefore you have to install the apps CalDAV-Sync and CardDav-Sync from the play store. Each application costs about 2 Euros. CardDav-Sync is also available for free but supports slightly less features. Both apps are implemented as sync adapters and therefore integrate seamlessly into the default contacts and calendar app of the Android os. For Android 4.1 and 4.4 the two additional apps JB Workaround CalDAV-Sync and JB Workaround CardDAV-Sync are required too. They only fix an Android bug, which otherwise would cause the deletion of the newly created CalDAV and CardDAV accounts after a software update.

Now we have to import the SSL certificate of the calendar server. To accomplish this, log into the server and convert the certificate to the binary format:

openssl x509 -in /etc/ssl/local/example.org.pem -outform DER -out /tmp/example.org.crt

Then copy it to the sd card of your smartphone, for example with adb:

adb push /tmp/example.org.crt /sdcard/

Lastly import the certificate by opening the android settings and navigate to “security”. Then choose “Install from SD card” and follow the import dialog.

Now you can create a new CalDAV account under Settings -> Add account -> CalDAV -> CalDAV. In the next dialog you must specify the server address in the format https://example.org/baikal/cal.php, the user name and the corresponding password. SSL is already activated. After the connection to the server is astablished, choose the calendars to synchronize and enter your email address. You also can deactivate the two-way synchronization, so that data is only synchronized from the server to the client. This avoids conflicts during synchronization but is not recommended if you also want to create new events on your smartphone.

Later on you can open the synchronization settings via Settings -> CalDAV -> Edit account settings or by starting the CalDAV-Sync App. Among others you can adapt the list of calendars to synchronize and the synchronization interval.

Now the configuration process is complete and you can view the synchronized apointments in the default calendar app. If you use the screen reader Talkback, I recommend to switch the layout of the calendar app to “Agenda”. To do so, open the combo box, which is located in the top-left corner of the calendar app and choose its last entry.

The configuration of the CardDAV synchronization is a similar process. You can create a new CardDAV account under Settings -> Add account -> CardDAV -> CardDAV. This time the server address is https://example.org/baikal/card.php. Beside that, use the same data like bfore. Then choose an address book and enter the email address again. The two-way synchronization for CardDAV entries is still marked as beta and therefore not activated by default. But if you also want to create new contacts on your smartphone, you should give it a try and activate it. I’ve encountered no problems so far.

If you want to add another address book, you have to perform the whole CardDAV account creation process again. Afterwards you can find your contacts data in the default contacts app. There you also can add new contacts and edit the existing ones.

Before you restart your phone, you have to launch the two Workaround-apps. This only must be done once.