Reposync
This program can create a local mirror though the yum/rpm mechanism itself, and does not have to rely on the rsync or curl method we use for syncing a repository. So this is purely for redhat flavour repositories.
It is not straightforward though...
separate configuration
This was needed at RINIS to gather various repositories onto the same server for internal installations. One thing you have to separate is the actual yum configuration of the mirror machine from the yum configuration of reposync. You rarely want the configuration of the repository machine to be the same as the one you are mirroring, for example you want to mirror different centos versions but only install packages for the centos version you are running.
To do this we created a separate directory :
| repotools |
|---|
| /opt/rinis/reposync/repotools
|
under which we create a separate cache, persistent and etc directory to use.
| directories |
|---|
| /opt/rinis/reposync/repotools/cache
/opt/rinis/reposync/repotools/etc
/opt/rinis/reposync/repotools/etc/yum.conf
/opt/rinis/reposync/repotools/etc/yum.conf.d/
/opt/rinis/reposync/repotools/persistdir
|
And in the etc directory we will put the same structure as normal. To point out where yum get's all directories fill yum.conf with these settings.
| yum.conf |
|---|
| [main]
cachedir=/opt/rinis/reposync/repotools/cache/$basearch/$releasever
persistdir=/opt/rinis/reposync/repotools/persistdir
reposdir=/opt/rinis/reposync/repotools/etc/yum.repos.d
keepcache=0
debuglevel=2
logfile=/opt/rinis/reposync/repotools/yum.log
exactarch=1
obsoletes=1
gpgcheck=1
plugins=1
installonly_limit=5
bugtracker_url=http://bugs.centos.org/set_project.php?project_id=19&ref=http://bugs.centos.org/bug_report_page.php?category=yum
distroverpkg=centos-release
assumeyes=1
|
The basis of this file is just copied from the regular yum.conf file, but some additions are :
- cachedir
- persistdir (for persistent files like gpg keys etc)
- reposdir (to maintain a separate set of .repo files)
- assumeyes=1 (for automated installation)
Of course we also need a .repo file, in this example it is 'passenger'.
| .repo file |
|---|
| [passenger.7.x86_64]
name=passenger
baseurl=https://oss-binaries.phusionpassenger.com/yum/passenger/el/7/x86_64
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://packagecloud.io/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
proxy=http://xsproxy.rinis.nl:3128
|
I am not sure if this is possible with one single file for multiple architectures/versions but this simple setup needs one repository file for each.
Now this turns out not to be enough to start a reposync session. Beware that the process relies on certain directories being created, like downloaded gpg key files. Always remember to run a yum update on this set of files as well !!. -c specifies the config file
| update |
|---|
| yum update -c /opt/rinis/reposync/repotools/etc/yum.conf
|
This will update the directories and sets you up for a synchronization :
| run reposync |
|---|
| reposync --config=/opt/rinis/reposync/repotools/etc/yum.conf --newest-only --repoid=passenger.7.x86_64 --download_path=/var/www/html/centos/repo/passenger
|
Now this will end up in a directory called the same name as the section, so we solved that with a symbolic link to the wanted structure. But we need to create one level of directory deeper first :
| create directory |
|---|
| mkdir -p /var/www/html/centos/repo/passenger/6
ln -f -s /var/www/html/centos/repo/passenger/passenger.6.i386 /var/www/html/centos/repo/passenger/6/i386
|
Sadly this does not recreate the metadata, but this is a trick we already know :
| createrepo |
|---|
| createrepo /var/www/html/centos/repo/passenger/6/i386
|
Done!! In the case of passenger with three repo files for 6/i386 6.x86_64 and 7/x86_64 we now have a directory structure :
| tree |
|---|
| passenger
|-- 6
| |-- i386 -> /var/www/html/centos/repo/passenger/passenger.6.i386
| `-- x86_64 -> /var/www/html/centos/repo/passenger/passenger.6.x86_64
|-- 7
| `-- x86_64 -> /var/www/html/centos/repo/passenger/passenger.7.x86_64
|-- noarch
| |-- nginx-filesystem-1.8.1-8.p5.0.24.el7.noarch.rpm
| |-- nginx-filesystem-1.8.1-8.p5.0.25.el6.noarch.rpm
| |-- nginx-filesystem-1.8.1-8.p5.0.25.el7.noarch.rpm
| |-- passenger-doc-5.0.24-8.el7.noarch.rpm
| |-- passenger-doc-5.0.25-8.el6.noarch.rpm
| `-- passenger-doc-5.0.25-8.el7.noarch.rpm
|-- passenger.6.i386
| |-- mod_passenger-5.0.25-8.el6.i686.rpm
| |-- nginx-1.8.1-8.p5.0.25.el6.i686.rpm
| |-- nginx-debuginfo-1.8.1-8.p5.0.25.el6.i686.rpm
| |-- passenger-5.0.25-8.el6.i686.rpm
| |-- passenger.6.i386 -> /var/www/html/centos/repo/passenger/passenger.6.i386
| |-- passenger-debuginfo-5.0.25-8.el6.i686.rpm
| |-- passenger-devel-5.0.25-8.el6.i686.rpm
| `-- repodata
| |-- 0a3cad7543896730719d41e1315bc109241058a0b2792d0a36583026cf1a7066-other.xml.gz
| |-- 46df39b88b9fda3a82bda25da6658290d1308f06fa2710d3a8028b8f02be88b3-primary.xml.gz
| |-- ade94d30a923a2a3b71e33fd773c1ab5e6530ebd19021a9f4ed421b913fe72c6-filelists.sqlite.bz2
| |-- b49ff861b6f84acbf623beee79338bc4118cd9e23f58160bc982e7dd119441ac-other.sqlite.bz2
| |-- c44cc1d67a8eb3d20f7654c40eea967c8aff49a1461b75a91c82e8e19c238ef1-filelists.xml.gz
| |-- f608bfa336732037ed3ad10089e923d2f3a9a6a4391e4679fe0430326db0d5f2-primary.sqlite.bz2
| `-- repomd.xml
|-- passenger.6.x86_64
| |-- mod_passenger-5.0.25-8.el6.x86_64.rpm
| |-- nginx-1.8.1-8.p5.0.25.el6.x86_64.rpm
| |-- nginx-debuginfo-1.8.1-8.p5.0.25.el6.x86_64.rpm
| |-- passenger-5.0.25-8.el6.x86_64.rpm
| |-- passenger.6.x86_64 -> /var/www/html/centos/repo/passenger/passenger.6.x86_64
| |-- passenger-debuginfo-5.0.25-8.el6.x86_64.rpm
| |-- passenger-devel-5.0.25-8.el6.x86_64.rpm
| `-- repodata
| |-- 20b66d1da80d4137fad24f3c5f7ce6babffa97f8add85d87d468ff377725d014-primary.sqlite.bz2
| |-- 2ffc04332cbd0dcb700d1fbca3739d44d3cd57c679b1fac12bd46500093e31b3-primary.xml.gz
| |-- 935b833e22f9eda46c85d1afac48457c4e6b8923b440331591be7b3835cea11f-filelists.xml.gz
| |-- b41b03c3209b5924f9fbf0f5a7a05230f4f68e7ab4859fbb8ae3e2e61e0dd713-other.sqlite.bz2
| |-- b713fe795d9e0f2311d177afb0646f97d683920c6393dac5c2313acaf2ae6ff0-other.xml.gz
| |-- c465aecba8a42bef40f5cc25ae4cacce97af36f1df4d2fc32bd97b2974c5b245-filelists.sqlite.bz2
| `-- repomd.xml
`-- passenger.7.x86_64
|-- mod_passenger-5.0.24-8.el7.x86_64.rpm
|-- mod_passenger-5.0.25-8.el7.x86_64.rpm
|-- nginx-1.8.1-8.p5.0.24.el7.x86_64.rpm
|-- nginx-1.8.1-8.p5.0.25.el7.x86_64.rpm
|-- nginx-debuginfo-1.8.1-8.p5.0.24.el7.x86_64.rpm
|-- nginx-debuginfo-1.8.1-8.p5.0.25.el7.x86_64.rpm
|-- passenger-5.0.24-8.el7.x86_64.rpm
|-- passenger-5.0.25-8.el7.x86_64.rpm
|-- passenger.7.x86_64 -> /var/www/html/centos/repo/passenger/passenger.7.x86_64
|-- passenger-debuginfo-5.0.24-8.el7.x86_64.rpm
|-- passenger-debuginfo-5.0.25-8.el7.x86_64.rpm
|-- passenger-devel-5.0.24-8.el7.x86_64.rpm
|-- passenger-devel-5.0.25-8.el7.x86_64.rpm
`-- repodata
|-- 1af7954f4bcfa5fa20e7189167b7f8ba924b16e5e2aa131e050746708474bf35-filelists.sqlite.bz2
|-- 288154ebcda37a279c8834424652f8aa272723e0bf80d08b0072e353cfaa49ae-filelists.xml.gz
|-- 2f014ffd6c5114f60cbff8539f8c5bfb8513e0b3155b9e9d122ef33756ce3295-other.sqlite.bz2
|-- 5c0fcf2aebed97d28271d6bff51ecf152c63e1150e18e364ba484306b8e9a255-primary.sqlite.bz2
|-- 5e65981bea284dc7cd43212cdb4fe14c9b1823475ab83268e9d4e608d2b86800-other.xml.gz
|-- 7a07d1d624143abb8546cd9e51ab27fa515c6da92e6cff19c5cb34d0a692346b-primary.xml.gz
`-- repomd.xml
15 directories, 51 files
|