use pigz instead of gzip

[seteqsystems]

Hi there,

Is there an option to have the backend using pigz instead of gzip for creating and installing opsi-packages?
pigz is a compatible gzip variant which uses more than just one cpu core.

We have pretty large packages and it would be very nice to speed up the package creation and installation process.

I already found a reference in Product.py to OPSI.Util.File.Archive
I can change System.which('gzip') to System.which('pigz') in /usr/share/pyshared/OPSI/Util/File/Archive/__init__.py

I just have NO experience in using Python so can you please show me how to recompile these changes?

Thanks

[mdecker]

Hi,

python usually does not need compiling. So if the commandline arguments of your alternative packer are compatible, your change might already be enough. Did you try it?

[ueluekmen]

Hi,

mdecker is right, you don't need to recompile anything. But you have to restart the opsiconfd-Daemon after your modification.

[seteqsystems]

Hi,

Thank you very much for your quick answers!
I already tried this and restarted both opsiconfd and opsipxeconfd ^^
The issue seemed more complicated, as gzip itself is invoked via the tar --gunzip parameter whenever a package is installed using the opsi-package-manager

For everyone who also needs this:

1. Edit the file /usr/share/pyshared/OPSI/Util/File/Archive/__init__.py (be sure to make a copy before editing!)
Search-Replace: "zcat" --> "pigzcat"
Search-Replace: "--gunzip" --> "--use-compress-program=pigz"
Search-Replace: "gzip" --> "pigz"

2. Download and compile pigz from here - Be sure to use the latest version (with support for the --rsyncable option) and not an operating system .deb or .rpm package!
Copy the pigz executable to /usr/bin

3. Copy /bin/zcat to /bin/pigzcat - edit /bin/pigzcat and replace gzip with pigz

4. Restart opsiconfd to activate the changes & have fun

[EDIT] If you have any questions - just ask!

[mdecker]

Only a small suggestion: It might be helpful to create a small patch file (for example via the diff command) of the changes to you made to the python code.

[seteqsystems]

As I am using an older opsi version at the moment I just provided the information on how to do it.
I don't know if the file has changed in between...

best regards

[seteqsystems]

Some more notes regarding this topic:

There is a problem with compiling the newest pigz 2.3 on ubuntu 12.04, the previous version 2.2.5 is compiling fine.
You just need to use a version >2.2.3 (because of the --rsyncable parameter support)

Caution when replacing "gzip" with "pigz", be sure to just replace the System.which parameters.

I created a diff for version 4.0.2.4 of the opsiconfd:

Code: Alles auswählen

237c237
<                               options += u'--gunzip'
---
>                               options += u'--use-compress-program=pigz'
259c259
<                               options += u'--gunzip'
---
>                               options += u'--use-compress-program=pigz'
302c302
<                               command += ' | %s --rsyncable' % System.which('gzip')
---
>                               command += ' | %s --rsyncable' % System.which('pigz')
323c323
<                               cat = System.which('zcat')
---
>                               cat = System.which('pigzcat')
345c345
<                               cat = System.which('zcat')
---
>                               cat = System.which('pigzcat')
395c395
<                               command += ' | %s --rsyncable' % System.which('gzip')
---
>                               command += ' | %s --rsyncable' % System.which('pigz')

[n.wenselowski]

Hello,

just to let you know: the upcoming version of OPSI (4.0.4.1) will feature pigz-support out-of-the-box if a corresponding version of pigz is installed.
No more need for hacks


Kind regards

N. Wenselowski