Skip to content

osrm build

mkdir strap
cd strap
git clone https://github.com/Project-OSRM/osrm-backend.git
sudo ldconfig /usr/local/lib && \
    git clone --branch v2021.3.0 --single-branch https://github.com/oneapi-src/oneTBB.git && \
    cd oneTBB && \
    mkdir build && \
    cd build && \
    cmake -DTBB_TEST=OFF -DCMAKE_BUILD_TYPE=Release ..  && \
    cmake --build . && \
    cmake --install .
cd ../osrm-backend
mkdir build
cd build
cmake -DENABLE_LTO=On ..
make

working solution bullseye

!!! "Use docker/Dockerfile"

Stick with bullseye until osrm provides a Dockerfile that has the new version and then copy that steps. !!

That is how this chapter was conceived. The steps needed to compile the whole osrm set (with some warnings along the way) is here

sudo su

I did everything as root, just do it also or comment on how it works without !

apt-get -y --no-install-recommends install ca-certificates cmake make git gcc g++ libbz2-dev libxml2-dev wget \
    libzip-dev libboost1.74-all-dev lua5.4 liblua5.4-dev pkg-config -o APT::Install-Suggests=0 -o APT::Install-Recommends=0

That takes care of all preliminaries also with some specific version preventing errors (like lua !). boost 1.74 will produce deprecation warnings, but they are from with libboost itself so it is NOT your fault.

ldconfig /usr/local/lib && \
    git clone --branch v2021.3.0 --single-branch https://github.com/oneapi-src/oneTBB.git && \
    cd oneTBB && \
    mkdir build && \
    cd build && \
    cmake -DTBB_TEST=OFF -DCMAKE_BUILD_TYPE=Release ..  && \
    cmake --build . && \
    cmake --install .

Still as root, this command will compile the tbb library in a way that does not crash the osrm-build. Probably it is best to uninstall any previous tbb version but i did not.

cd ../../osrm-backend
mkdir build
cd build
cmake ..
make

This very simplified command from the Docker version works, you could play around with some options like :

  • -DCMAKE_BUILD_TYPE=Debug|Release
  • -DENABLE_ASSERTIONS=On|Off
  • -DBUILD_TOOLS=On|Off
  • -DENABLE_LTO=Off|On

Build tools is default on, and it probably always should be or you can't build road network. link time optimizations gave me a lot of headache so you could toggle that to see what gives.

With all defaults it does produce a working toolset.

make install

Will put everything in /usr/local, so the library and tools would be :

/usr/local/lib/libosrm.a
...
/usr/local/bin/osrm-extract
/usr/local/bin/osrm-partition
/usr/local/bin/osrm-customize
/usr/local/bin/osrm-contract
/usr/local/bin/osrm-datastore
/usr/local/bin/osrm-routed
...
/usr/local/include/osrm/osrm.hpp

Compiling this into backend :

cd ~/projects/klopt/solver/backend
make

First problem is :

/usr/local/include/osrm/engine/engine_config.hpp:31:10: fatal error: 'storage/storage_config.hpp' file not found

Is storage_config.hpp gone ? No it is in :

/usr/local/include/osrm/storage/

So specifying a better OSRM_DIR would help :

If you get multiple definitions of json_renderer functions, it usually originated from the file fcgi.cpp.

This worked for me, comment out in backend.hpp

//#include "util/json_renderer.hpp

Seemed to do the trick. It now only get's included in osrm.cpp

#include "util/json_renderer.hpp"

Which is the only place where it is used. No single clue why that worked before !!

troubleshooting

Below are a couple of tips for getting passed certain problems. Note however that the chapter above provides a working compilation !

Problems with tbb :

https://github.com/Project-OSRM/osrm-backend/issues/6264

and

https://github.com/Project-OSRM/osrm-backend/issues/6253

These seems to be solved by using mason or conan.

This will use a pre-built TBB library that's known to work, rather than system-installed libraries.

In the old version

cmake . -B build -DENABLE_MASON=ON

But this still leaves us with lua problems. 

/home/kees/projects/klopt/base/backend-build/osrm-backend/third_party/sol2/sol2/sol.hpp:6909:40: error: call of overloaded ‘push<const char32_t*>(lua_State*&, const char32_t [2], const char32_t*)’ is ambiguous
 6909 |     return stack::push<const char32_t*>(L, str, str + sz);
      |            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~

It seems to be a tightening of the rules in gcc that causes this. But advice of going back to gcc9 just do nothing.

note :

In the build below, it still says somewhere along the build :

[1] gcc-10

So it takes the highest gcc version it can find ? I think it would be wise to freeze system + sources to maintain a stable version. Than try to upgrade, but only if it takes not too much effort.

So i tried the latest version of osrm.

However in the latest osrm, they changed mason to conan ?! Mason seems to be unsupported recently. Also .. april 2023 : conan is now version 2, but they have not ported all recipes yet. So altered pip command to the last V1 :

sudo pip install conan==1.59
conan profile detect
vim CMakeList
cmake . -B build -DENABLE_CONAN=ON
cd build
make

This does seem to work (with some warnings), so for now we stick with debian bullseye (11) end the newer osrm version. If all works out !.

Make a new bullseye branch directory for that ! Maybe alter osrm-backend to buster ? (try on buster VM)

spirit/include/phoenix.hpp deprecated

Though not an error, it is still annoying.

The message reads : 

/home/kees/.conan/data/boost/1.79.0/_/_/package/c731e192b7eb90f64ffc78cb4e802cfd5d59d15b/include/boost/spirit/include/phoenix.hpp:12:1: note: ‘#pragma message: This header is deprecated. Use <boost/phoenix.hpp> instead.’

dockerfile

Since all paths fail, i took a look at the Dockerfile inside osrm-backend/docker. A lot of useful information is in there. For instance:

FROM debian:bullseye-slim as builder

Means it can be done on bullseye !

preliminaries

First it installs the preliminaries.

sudo apt-get update && apt-get -y --no-install-recommends install ca-certificates cmake make git gcc g++ libbz2-dev libxml2-dev wget libzip-dev libboost1.74-all-dev lua5.4 liblua5.4-dev pkg-config -o APT::Install-Suggests=0 -o APT::Install-Recommends=0

We get final hints about which package will work. and indeed libboost1.74 seems to be good.

This step finished without problems.

oneTBB

Indeed where I ended up installing from was oneTBB, but the dockerfile is much more detailed.

First remove the old libtbb-dev installation

apt-get remove --purge libtbb-dev
If it removes to much, rerun the preliminaries again.

git clone --branch v2021.3.0 --single-branch https://github.com/oneapi-src/oneTBB.git
cd oneTBB
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr -DTBB_TEST=OFF -DCMAKE_BUILD_TYPE=Release ..
cmake --build .
sudo cmake --install .

Maybe the Release build could be Debug, but we stick with this. The default install prefix on linux is /usr/local, but osrm-backend does not seem to look there. So we alter it with CMAKE_INSTALL_PREFIX.

This Works !!, probably the TBB_TEST flag did the trick !

osrm

The Dockerfile does a copy now 

COPY . /src
WORKDIR /src

But there is a file called hook/build that explains it really wants the repo root. So the osrm steps should be done from one directory higher.

cd ../../osrm-backend

Then build.

mkdir -p build
cd build
cmake .. -DCMAKE_BUILD_TYPE="Debug" -DENABLE_ASSERTIONS="Off" -DBUILD_TOOLS="On" -DENABLE_LTO=On && make -j 8 install

The docker command tries to figure out all the parameters so this is the shortened version. We first try it with lto (link time optimization) but since we had so much trouble with that set it to off as soon as we get into problems.

boost 1.79

This seems to be needed because 1.74 (the default on debian bullseye) gives errors on path::append_v3() calls. And indeed these are only available in 1.79.

Quick setup : from https://stackoverflow.com/questions/12578499/how-to-install-boost-on-ubuntu

wget -O boost_1_79_0.tar.gz https://sourceforge.net/projects/boost/files/boost/1.79.0/boost_1_79_0.tar.gz/download
tar zxvf boost_1_79_0.tar.gz
cd boost_1_79_0
./bootstrap.sh --prefix=/usr/
./b2
sudo ./b2 install

Now you can link and run the backend. This was all very confusing, so in the next chapter we will try to make a complete install script from within 1 directory.

complete script

Attempt to build everything from the ground up, front and backend, database. After that attempt to add a java backend as well, with jsprit and graphopper.

Add this script to the klopt repository !. For now we done this from

/work/projects/klopt/new/base, docker-compose is in there and works.

git@bitbucket.org:keesklopt/klopt

This seems like an older tree, with both backend and frontend inside. The changes just point at solver and planner being the new 'split' front and backend trees. So it would be best to rename the develop branch to split and make develop and main follow a new structure.

  • add the docker-compose.yaml file from base
  • add a makefile to build the entire application

The makefile should clone the parts so it remains sure where all the newer versions come from.

graphopper

First try to make this work and then compare the speed with osrm.

jsprit

Try out the VRP algorithms and compare them with the lkm algorithm.

lkh

First try out the own implementation, but if it gives any problems, put the original in there as well.

API

Create an API for running any algorithm needed. Start with cheapest insert !