forum.opsi.org

Hi, I am trying to debug a problem with PXE Boot not working.

I get the following error on an UEFI 64bit machine with secure boot disabled:

Code: Alles auswählen

tftp://<private-ip>/linux/pxelinux.cfg/shimx64.efi.signed... ok
shimx64.efi.signed : 1191448 bytes [EFI]
dppath: shimx64.efi.signed
path: shimx64.efi.signed
Fetching Netboot Image
Malformed binary after Attribute Certificate Table
datasize: ? SumOfBytesHashed: ? SecDir->Size ?
hashsize: ? SecDir->VirtualAddress: 0x0034B8A18
Failed to load image: Invalid Parameter
start_image() returned Invalid Parameter
Could not boot image: Error 0x7f048282 (http://ipxe.org/7f048282)
No more network devices

I have been trying to fix this for days to no avail. The BIOS of the client is on the latest version. Maybe somebody here knows what is happening here or can help me debug this?

See below for more information:

Opsi server version: 4.2 on Debian 10 with opsi-server-full package installed (opsi-tftpd-hpa is installed as well)

Log file /var/log/opsi/opsipxeconfd.log:

Code: Alles auswählen

[5] [2021-12-15 13:50:38.306] [Opsipxeconfd   ] Got connection from client   (opsipxeconfd.py:231)
[3] [2021-12-15 13:50:38.315] [               ] Cannot use more than one pxe config template, got: /tftpboot/linux/pxelinux.cfg/install-grub-x64, install3264   (opsipxeconfd.py:773)
[5] [2021-12-15 13:50:38.315] [               ] Did not find any alternate UEFI pxeConfigTemplate, will use the default UEFI template   (opsipxeconfd.py:782)
[5] [2021-12-15 13:50:38.322] [               ] UEFI GRUB configuration detected for redacted.domain.de   (pxeconfigwriter.py:146)
[5] [2021-12-15 13:50:38.326] [               ] PXE boot configuration for host redacted.domain.de is now set at '/tftpboot/linux/pxelinux.cfg/01-a4-4c-c8-14-47-67'   (opsipxeconfd.py:630)
(Nothing after this, even when starting PXE on the client)

Config file /etc/opsi/opsipxeconfd.conf:

Code: Alles auswählen

backend config dir = /etc/opsi/backends
dispatch config file = /etc/opsi/backendManager/dispatch.conf
pid file = /var/run/opsipxeconfd/opsipxeconfd.pid
log file = /var/log/opsi/opsipxeconfd.log
log level = 5
log format = %(log_color)s[%(opsilevel)d] [%(asctime)s.%(msecs)03d]%(reset)s %(message)s   (%(filename)s:%(lineno)d)
pxe config dir = /tftpboot/linux/pxelinux.cfg
pxe config template = /tftpboot/linux/pxelinux.cfg/install
uefi netboot config template x86 = /tftpboot/linux/pxelinux.cfg/install-elilo-x86
uefi netboot config template x64 = /tftpboot/linux/pxelinux.cfg/install-grub-x64
max control connections = 5
max pxe config writers = 100

DHCP Settings (pfsense as DHCP, suppose that 10.10.1.1 is the IP of the OPSI server):

Code: Alles auswählen

tftp-server: 10.10.1.1
next-server: 10.10.1.1
Standard BIOS filename: linux/pxelinux.0
UEFI 32Bit Filename: linux/pxelinux.cfg/elilo-x86.efi
UEFI 64Bit Filename: linux/pxelinux.cfg/shimx64.efi.signed
root-path: empty (I also tried using /tftpboot, but same error)

Client settings:

Code: Alles auswählen

UEFI Boot: enabled
clientconfig.dhcpd.filename: linux/pxelinux.cfg/elilo.efi

I followed the breaking change in Opsi 4.2 and changed the "uefi netboot config template x64" value in opsipxeconfd.conf as you can see above.
Also if I enable Secure boot, I instead get the following error: "Operating System Loader has no signature. Incompatible with SecureBoot."

Any ideas how I can debug this?

Hi

have you read this?

It seems you are using ipxe. As far as I know ipxe doesn't support secureboot at all, or at least they do not distrubute a signed binary.

what is the output in /var/log/syslog when the client boots?

Maybe this could also help

Regards
Mathias

m.radtke hat geschrieben: ↑15 Dez 2021, 14:46 have you read this?

Yes and even through the source code, this does not help at all.

m.radtke hat geschrieben: ↑15 Dez 2021, 14:46 As far as I know ipxe doesn't support secureboot at all, or at least they do not distrubute a signed binary.

I am not trying to use secureboot, I have secureboot disabled.

m.radtke hat geschrieben: ↑15 Dez 2021, 14:46 what is the output in /var/log/syslog when the client boots?

Code: Alles auswählen

Dec 15 13:51:08 srv-opsivm in.tftpd[8784]: RRQ from 10.10.1.2 filename linux/pxelinux.cfg/shimx64.efi.signed
Dec 15 13:51:11 srv-opsivm in.tftpd[8787]: RRQ from 10.10.1.2 filename linux/pxelinux.cfg/grubx64.efi

10.10.1.2 is the IP of the DHCP server (a pfsense firewall in this case).

m.radtke hat geschrieben: ↑15 Dez 2021, 14:46 Maybe this could also help

That was the first link I found when I searched myself. Unfortunately it didn't really help, or at least I don't think it applies to my case. The iPXE in my case is just an iPXE embedded in the UEFI of my client that is trying to PXE boot (which is a Dell Laptop) and is automatically started with the right parameters when I choose the IPv4 netboot option. I don't think I am able to get an iPXE shell in this case and therefore I am unable to run a custom iPXE command from there.

Try an opsi boot cd (from usb or something), to see if that works properly. Maybe it's not iPXE.

What kind of dell laptop is this?

I am having a hard time understanding what is happening here:

Code: Alles auswählen

Dec 15 13:51:08 srv-opsivm in.tftpd[8784]: RRQ from 10.10.1.2 filename linux/pxelinux.cfg/shimx64.efi.signed
Dec 15 13:51:11 srv-opsivm in.tftpd[8787]: RRQ from 10.10.1.2 filename linux/pxelinux.cfg/grubx64.efi

10.10.1.2 is the IP of the DHCP server (a pfsense firewall in this case).

The RRQ should come from the Client, not from the DHCP-Server...

SisterOfMercy hat geschrieben: ↑16 Dez 2021, 15:36 Try an opsi boot cd (from usb or something), to see if that works properly. Maybe it's not iPXE.

What kind of dell laptop is this?

There is no need to test an Opsi boot CD, I had iPXE netboot working with Opsi on these Dell Latitude 5480 before. I cannot get this to work after updating to version 4.2.

ThomasT hat geschrieben: ↑16 Dez 2021, 16:42 I am having a hard time understanding what is happening here:
The RRQ should come from the Client, not from the DHCP-Server...

I think this is just address translation doing something weird. The DHCP-Server is also our firewall. The opsi server and the client are in different networks (I translated them to simple 10.10.1.x IPs in order to not overcomplicate this issue, since network configuration is not the issue here, because it was working before with this exact Firewall setup), but still a little weird.
Note that the Dell laptop seems to be getting the shimx64.efi.signed just fine, because the size it displays (1191448 bytes) during PXE boot matches perfectly with the size of shimx64.efi.signed on the opsi server. This is why I don't think that the DHCP (which is also the Firewall and Gateway) appearing as RRQ is an issue.

To be fair I can sometimes get the IP of a client to popup in the log, but that is another different testing szenario. If I use a Xen Server VM in a different network as a client with IP 10.10.4.202 (instead of the Dell laptop), I get the following to popup:

Code: Alles auswählen

Dec 16 17:15:12 srv-opsivm in.tftpd[68875]: RRQ from 10.10.4.202 filename linux/pxelinux.0
Dec 16 17:15:15 srv-opsivm in.tftpd[68876]: RRQ from 10.10.4.202 filename linux/pxelinux.cfg/beb0d1f6-cbd3-6072-7306-06335e238f21
Dec 16 17:15:16 srv-opsivm in.tftpd[68877]: RRQ from 10.10.4.202 filename linux/pxelinux.cfg/01-ea-79-5d-c9-86-ab
Dec 16 17:15:17 srv-opsivm in.tftpd[68878]: RRQ from 10.10.4.202 filename linux/linux
Dec 16 17:15:17 srv-opsivm in.tftpd[68879]: RRQ from 10.10.4.202 filename linux/linux.cbt
Dec 16 17:15:17 srv-opsivm in.tftpd[68880]: RRQ from 10.10.4.202 filename linux/linux.0
Dec 16 17:15:17 srv-opsivm in.tftpd[68881]: RRQ from 10.10.4.202 filename linux/linux.com
Dec 16 17:15:17 srv-opsivm in.tftpd[68882]: RRQ from 10.10.4.202 filename linux/linux.c32

And /var/log/opsi/opsipxeconfd.log shows the following:

Code: Alles auswählen

[3] [2021-12-16 17:14:52.132] [               ] Cannot use more than one pxe config template, got: /tftpboot/linux/pxelinux.cfg/install-grub-x64, install3264   (opsipxeconfd.py:773)
[5] [2021-12-16 17:14:52.132] [               ] Did not find any alternate UEFI pxeConfigTemplate, will use the default UEFI template   (opsipxeconfd.py:782)
[5] [2021-12-16 17:14:52.141] [               ] UEFI GRUB configuration detected for opsi-test   (pxeconfigwriter.py:146)
[5] [2021-12-16 17:14:52.143] [               ] PXE boot configuration for host opsi-test is now set at '/tftpboot/linux/pxelinux.cfg/01-ea-79-5d-c9-86-ab'   (opsipxeconfd.py:630)
[5] [2021-12-16 17:15:16.174] [PXEConfigWriter] Pipe '/tftpboot/linux/pxelinux.cfg/01-ea-79-5d-c9-86-ab' opened, piping pxe boot configuration   (pxeconfigwriter.py:199)

Unfortunately the client then shows a different error on Netboot:

Code: Alles auswählen

tftp://10.10.1.1/linux/pxelinux.0... ok

PXELINUX 3.71 2008-07-31 Copyright (C) 1994-2008 H. Peter Anvin
UNDI data segment at: 0009B340
UNDI data segment size: 2CB8
UNDI code segment at: 0009ABF0
UNDI code segment size: 074C
PXE entry point found (we hope) at 9ABF:0307
Getting cached packet 01
Getting cached packet 02
Getting cached packet 03
My IP address seems to be 0A0A04CA 10.10.4.202
ip=10.10.4.202:10.10.1.1:10.10.4.1:255.255.255.0
TFTP prefix: linux/
Trying to load: pxelinux.cfg/beb0d1f6-cbd3-6072-7306-06335e238f21
Trying to load: pxelinux.cfg/01-ea-79-5d-c9-86-ab
Unknown keyword in configuration file: set
Unknown keyword in configuration file: menuentry
Unknown keyword in configuration file: set
Unknown keyword in configuration file: (pxe)/linux/install-x64
Missing parameter in configuration file.
Could not find kernel image: linux
boot: _ (here it expects user input)

I guess this error might also be due to old firmware, that is why I kept testing only with the Dell Laptop, where I can't even seem to get past the shimx64.efi.signed.

i-love-opsi hat geschrieben: ↑16 Dez 2021, 17:39 There is no need to test an Opsi boot CD, I had iPXE netboot working with Opsi on these Dell Latitude 5480 before. I cannot get this to work after updating to version 4.2.

But you can never be sure everything is the same. If you updated the bios of those dells maybe _something_ has changed. You need to somehow rule out various things, so you're not looking in the wrong direction.

i-love-opsi hat geschrieben: ↑16 Dez 2021, 17:39 Unfortunately the client then shows a different error on Netboot:

tftp://10.10.1.1/linux/pxelinux.0... ok
PXELINUX 3.71 2008-07-31 Copyright (C) 1994-2008 H. Peter Anvin

I don't know what kind of error is from opsipxeconfd, but you have created an UEFI pipe and try to boot this with a legacy client.

i-love-opsi hat geschrieben: ↑16 Dez 2021, 17:39
Unfortunately the client then shows a different error on Netboot:

Code: Alles auswählen

tftp://10.10.1.1/linux/pxelinux.0... ok

PXELINUX 3.71 2008-07-31 Copyright (C) 1994-2008 H. Peter Anvin
UNDI data segment at: 0009B340
UNDI data segment size: 2CB8
UNDI code segment at: 0009ABF0
UNDI code segment size: 074C
PXE entry point found (we hope) at 9ABF:0307
Getting cached packet 01
Getting cached packet 02
Getting cached packet 03
My IP address seems to be 0A0A04CA 10.10.4.202
ip=10.10.4.202:10.10.1.1:10.10.4.1:255.255.255.0
TFTP prefix: linux/
Trying to load: pxelinux.cfg/beb0d1f6-cbd3-6072-7306-06335e238f21
Trying to load: pxelinux.cfg/01-ea-79-5d-c9-86-ab
Unknown keyword in configuration file: set
Unknown keyword in configuration file: menuentry
Unknown keyword in configuration file: set
Unknown keyword in configuration file: (pxe)/linux/install-x64
Missing parameter in configuration file.
Could not find kernel image: linux
boot: _ (here it expects user input)

I guess this error might also be due to old firmware, that is why I kept testing only with the Dell Laptop, where I can't even seem to get past the shimx64.efi.signed.

Thats becasue this is a BIOS Client and you did enable UEFI Netboot in the opsi-configed.
Those named pipes aren't compatible with eachother.

I would suggest to try what SisterOfMercy proposed. Try the boot-cd to be sure.

One additional thought about the malformed binary stuff: We once had a notebook from a customer that didn't want to boot with shim/grub but did boot with elilo.
Thing was it had to enable other OS booting in the UEFI. Until then it was restricted to boot Windows only, but don't know what elilo did to make this work.

Regards
Mathias

SisterOfMercy hat geschrieben: ↑20 Dez 2021, 01:11
But you can never be sure everything is the same. If you updated the bios of those dells maybe _something_ has changed. You need to somehow rule out various things, so you're not looking in the wrong direction.

I just tried the Opsi Live CD and it works without problems. So the problem must be something with PXE and/or with the shimx64 bootloader. Is there any way to go back to elilo instead of shimx64 for Opsi 4.2? If not, what can I do to debug the PXE problem further?

SisterOfMercy hat geschrieben: ↑20 Dez 2021, 01:11 I don't know what kind of error is from opsipxeconfd, but you have created an UEFI pipe and try to boot this with a legacy client.

OK it seems that the UEFI option for Xen VMs is not working, so I can only debug this problem with real hardware (the Dell laptop) in the future. Unfortunately then, the iPXE error message from my first post is still the error that I am getting.

Maybe the problem is not in shimx64.efi.signed but in grubx64.efi, as the last RRQ in /var/log/syslog shown is linux/pxelinux.cfg/grubx64.efi (after linux/pxelinux.cfg/shimx64.efi.signed).

The documentation says that the DHCP PXE UEFI 64bit filename should be linux/pxelinux.cfg/shimx64.efi.signed. If I don't do as the documentation says, and use linux/pxelinux.cfg/grubx64.efi as PXE filename, then it works (although it is stuck for 2 minutes at the beginning of the progress):

Code: Alles auswählen

Dec 22 13:51:12 srv-opsivm in.tftpd[116936]: RRQ from 10.10.1.2 filename linux/pxelinux.cfg/grubx64.efi
Dec 22 13:51:14 srv-opsivm in.tftpd[116937]: RRQ from 10.10.1.2 filename /grub/x86_64-efi/command.lst
Dec 22 13:51:17 srv-opsivm in.tftpd[116938]: RRQ from 10.10.1.2 filename /grub/x86_64-efi/fs.lst
Dec 22 13:51:19 srv-opsivm in.tftpd[116939]: RRQ from 10.10.1.2 filename /grub/x86_64-efi/crypto.lst
Dec 22 13:51:21 srv-opsivm in.tftpd[116941]: RRQ from 10.10.1.2 filename /grub/x86_64-efi/terminal.lst
Dec 22 13:51:23 srv-opsivm in.tftpd[116944]: RRQ from 10.10.1.2 filename /grub/grub.cfg
Dec 22 13:51:26 srv-opsivm in.tftpd[116947]: RRQ from 10.10.1.2 filename /linux/pxelinux.cfg/01-10-65-30-24-97-1f
Dec 22 13:51:28 srv-opsivm in.tftpd[116949]: RRQ from 10.10.1.2 filename /linux/pxelinux.cfg/01-10-65-30-24-97-1f
Dec 22 13:51:32 srv-opsivm in.tftpd[116952]: RRQ from 10.10.1.2 filename /linux/install-x64
Dec 22 13:51:36 srv-opsivm in.tftpd[116953]: RRQ from 10.10.1.2 filename /linux/miniroot-x64

Maybe you can update your documentation or is this just an unreliable workaround that I just did?

forum.opsi.org

PXE boot fails with "Malformed binary"

PXE boot fails with "Malformed binary"

Re: PXE boot fails with "Malformed binary"

Re: PXE boot fails with "Malformed binary"

Re: PXE boot fails with "Malformed binary"

Re: PXE boot fails with "Malformed binary"

Re: PXE boot fails with "Malformed binary"

Re: PXE boot fails with "Malformed binary"

Re: PXE boot fails with "Malformed binary"

Re: PXE boot fails with "Malformed binary"

Re: PXE boot fails with "Malformed binary"