Seite 1 von 1
use pigz instead of gzip
Verfasst: 05 Feb 2013, 12:02
von seteqsystems
Hi there,
Is there an option to have the backend using pigz instead of gzip for creating and installing opsi-packages?
pigz is a compatible gzip variant which uses more than just one cpu core.
We have pretty large packages and it would be very nice to speed up the package creation and installation process.
I already found a reference in Product.py to OPSI.Util.File.Archive
I can change System.which('gzip') to System.which('pigz') in /usr/share/pyshared/OPSI/Util/File/Archive/__init__.py
I just have NO experience in using Python so can you please show me how to recompile these changes?
Thanks
Re: use pigz instead of gzip
Verfasst: 05 Feb 2013, 12:34
von mdecker
Hi,
python usually does not need compiling. So if the commandline arguments of your alternative packer are compatible, your change might already be enough. Did you try it?
Re: use pigz instead of gzip
Verfasst: 05 Feb 2013, 12:51
von ueluekmen
Hi,
mdecker is right, you don't need to recompile anything. But you have to restart the opsiconfd-Daemon after your modification.
Re: use pigz instead of gzip
Verfasst: 05 Feb 2013, 13:49
von seteqsystems
Hi,
Thank you very much for your quick answers!
I already tried this and restarted both opsiconfd and opsipxeconfd ^^
The issue seemed more complicated, as gzip itself is invoked via the tar --gunzip parameter whenever a package is installed using the opsi-package-manager
For everyone who also needs this:
1. Edit the file /usr/share/pyshared/OPSI/Util/File/Archive/__init__.py (be sure to make a copy before editing!)
Search-Replace: "zcat" --> "pigzcat"
Search-Replace: "--gunzip" --> "--use-compress-program=pigz"
Search-Replace: "gzip" --> "pigz"
2. Download and compile pigz from here - Be sure to use the latest version (with support for the --rsyncable option) and not an operating system .deb or .rpm package!
Copy the pigz executable to /usr/bin
3. Copy /bin/zcat to /bin/pigzcat - edit /bin/pigzcat and replace gzip with pigz
4. Restart opsiconfd to activate the changes & have fun 
[EDIT] If you have any questions - just ask!
Re: use pigz instead of gzip
Verfasst: 05 Feb 2013, 14:14
von mdecker
Only a small suggestion: It might be helpful to create a small patch file (for example via the diff command) of the changes to you made to the python code.
Re: use pigz instead of gzip
Verfasst: 05 Feb 2013, 15:56
von seteqsystems
As I am using an older opsi version at the moment I just provided the information on how to do it.
I don't know if the file has changed in between...
best regards
Re: use pigz instead of gzip
Verfasst: 11 Mär 2013, 12:00
von seteqsystems
Some more notes regarding this topic:
There is a problem with compiling the newest pigz 2.3 on ubuntu 12.04, the previous version 2.2.5 is compiling fine.
You just need to use a version >2.2.3 (because of the --rsyncable parameter support)
Caution when replacing "gzip" with "pigz", be sure to just replace the System.which parameters.
I created a diff for version 4.0.2.4 of the opsiconfd:
Code: Alles auswählen
237c237
< options += u'--gunzip'
---
> options += u'--use-compress-program=pigz'
259c259
< options += u'--gunzip'
---
> options += u'--use-compress-program=pigz'
302c302
< command += ' | %s --rsyncable' % System.which('gzip')
---
> command += ' | %s --rsyncable' % System.which('pigz')
323c323
< cat = System.which('zcat')
---
> cat = System.which('pigzcat')
345c345
< cat = System.which('zcat')
---
> cat = System.which('pigzcat')
395c395
< command += ' | %s --rsyncable' % System.which('gzip')
---
> command += ' | %s --rsyncable' % System.which('pigz')
Re: use pigz instead of gzip
Verfasst: 25 Nov 2013, 16:28
von n.wenselowski
Hello,
just to let you know: the upcoming version of OPSI (4.0.4.1) will feature pigz-support out-of-the-box if a corresponding version of pigz is installed.
No more need for hacks
Kind regards
N. Wenselowski