Non ASCII characters

[mirkt]

by the way, I have accidentally noticed, that leaving whitespace(s) after

Code: Alles auswählen

encoding=utf8 

Does not set does not set proper encoding. Maybe it could?

[n.wenselowski]

Hi,

by the way, I have accidentally noticed, that leaving whitespace(s) after

Code: Alles auswählen

encoding=utf8 

Does not set does not set proper encoding. Maybe it could?

It should
I opened a ticket for this to be fixed.


Bye

Niko

[n.wenselowski]

Hi,

fix should be coming with opsi-script 4.11.6.4.


Kind regards

Niko

[mirkt]

Thank you for the fix

I am not sure if anyone else encountered this problem.. To make it easier I have written a little (and dirty) Python script which can set encoding to all your OPSI scripts (if file is not ASCII encoded).. It's for UTF-8 encoded scripts.

I can share it with you:

Code: Alles auswählen

#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""
This little script can search for UTF-8 encoded opsi scripts and set proper
encoding inside them.


If requiredWinstVersion IS defined in opsi script, this script replaces that line
to:
requiredWinstVersion >= "4.11.4.2"
and defines encoding on the next line.

If requiredWinstVersion IS NOT defined, this script defines encoding on the first
line of the file.


Usage examples:
./add_encoding.py -d PATH_TO_PACKAGE_OR_WORKBENCH -l
    => lists UTF-8 encoded scripts where encoding is not defined

./add_encoding.py -d PATH_TO_PACKAGE_OR_WORKBENCH
    => sets encoding in UTF-8 encoded scripts
"""

import sys
import getopt
import os
import chardet
import re

opsiscripts_extentions = (".ins", ".opsiscript")
modify_files_with_encoding = "utf-8"
define_encoding = "encoding=utf8"
define_required_winst_version = 'requiredWinstVersion >= "4.11.4.2"'
re_encoding = re.compile("^\s*encoding\s*=.*", re.IGNORECASE)
re_required_version = re.compile("^\s*requiredWinstVersion\s*[<>]?=.*", re.IGNORECASE)

list_only = False
path = ""

opts, otherargs = getopt.getopt(sys.argv[1:], "d:l", ["directory=", "list-only"])

for opt, arg in opts:
    if opt in ("-d", "--directory"):
        if os.path.exists(arg):
            path = arg
        else:
            print >> sys.stderr, "Directory you specified doesn't exist"
            sys.exit(-1)

    elif opt in ("-l", "--list-only"):
        list_only = True

if not path:
    print >> sys.stderr, "Directory was not specified.."
    sys.exit(-1)

opsiscripts = []
for (path, dirs, files) in os.walk(path):
    for filename in files:
        if filename.endswith(opsiscripts_extentions):
            opsiscripts.append(os.sep.join([path, filename]))

opsiscripts_to_update = []
for opsiscript in opsiscripts:
    if (chardet.detect(open(opsiscript, "r").read())['encoding'] <> "ascii"):
        opsiscripts_to_update.append(opsiscript)

for opsiscript in opsiscripts_to_update:
    encoding_defined = False
    required_version_defined = False
    with open(opsiscript) as file:
        for line in file:
            if re_encoding.search(line):
                encoding_defined = True
            if re_required_version.search(line):
                required_version_defined = True

    if not encoding_defined:
        if list_only:
            print "* Encoding not defined in %s" % opsiscript
            continue

        print "* Modifying %s" % opsiscript
        file_content = open(opsiscript).readlines()
        if not required_version_defined:
            file_content.insert(0, define_encoding + "\n")
            open(opsiscript, "w").writelines(file_content)
        else:
            new_file_content = []
            line_counter = 1
            for line in file_content:
                if re_required_version.search(line):
                    new_file_content.insert(line_counter, define_required_winst_version + "\n")
                    line_counter += 1
                    new_file_content.insert(line_counter, define_encoding + "\n")
                else:
                    new_file_content.insert(line_counter, line)
                line_counter += 1
            open(opsiscript, "w").writelines(new_file_content)

I have "fixed" entire opsi_workbench by using this script..

You can backup of all *.ins, *.opsiscript files (before executing the script) by running:

Code: Alles auswählen

rsync -amv --include '*/' --include '*.ins' --include '*.opsiscript' --exclude '*' YOUR_WORKBENCH_PATH/ YOUR_BACKUP_PATH