1 Topic by dud225

  • dud225
  • Member
  • Offline
  • Registered: 2014-10-17
  • Posts: 12

Topic: GParted webbooting

Hello

I'm looking for booting GParted directly from the internet by leveraging iPXE capabilities. This way one no longer needs to retrieve any ISO file, iPXE is indeed able to download files from a webserver and handle them. If you don't know well that project I advise you to get an idea on its power by glancing at boot.salstar.sk. To test it with KVM you may use that command : qemu-system-x86_64 -enable-kvm -net user -net nic,model=virtio -m 512 then open a iPXE shell by typing CTRL+B, from that point type "dhcp" and finally "chain http://boot.salstar.sk".

Actually there are mainly 2 methods for booting a Linux distribution :
1) straight booting an ISO file through memdisk
2) configuring iPXE by providing a linux kernel, an initrd image and a squash fs

Unfortunately I'm unable to boot GParted as for now :

Method 1 doesn't work because Linux is unable to find the root fs. A lot of people are reporting this issue and highlighting that Clonezilla is nonetheless booting properly. After digging into this issue, I think that Clonezilla supports memdisk thanks to the phram module (more information could be found on syslinux website). Actually I found out an phram.ko module inside Clonezilla's but unfortunately this module is not provided by GParted. From what I saw the script 9990-main.sh called by the boot process is able to handle memdisk through the phram and mtdblock  modules.
Would it be possible to add kernel modules phram and mtdblock in the initrd file so that memdisk would be supported ?

Method 2 is simplier : the PXE configuration is the same as what is indicated on GParted's website (livepxe.php) except that all files are retrieved online. Despites this method is possible with Debian Live (all files are available here : live.debian.net/cdimage/release/stable/amd64/web/) it may be not used with GParted since it doesn't not publish the 3 needed files.
As a fallback scheme would it be possible to extract those files and host them on a public webserver ?

Thanks for your answer.

H. Werner

2 Reply by gedakc

  • gedakc
  • Developer
  • Offline
  • Registered: 2007-11-30
  • Posts: 4,593

Re: GParted webbooting

Thank you for your interest in GParted, and the comparison with Clonezilla booting.  I have notified our Live image maintainer about this post.

3 Reply by stevenshiau

  • stevenshiau
  • Member
  • Offline
  • Registered: 2008-04-02
  • Posts: 293

Re: GParted webbooting

Since I am one of the developers of Clonezilla, this problem looks weird to me. I use the same methods to create GParted live and Clonezilla live. They should have the same way, unless there is a bug.
Actually I have no problem to use your method 1 to boot GParted live. The PXELinux config I have for GParted live is:
======================
label GParted-live
  # MENU DEFAULT
  # MENU HIDE
  MENU LABEL GParted Live iso
  # MENU PASSWD
  kernel memdisk
  initrd gparted-live-0.19.1-4-i686-pae.iso
  append iso raw

  TEXT HELP
  GParted Live iso
  ENDTEXT
======================
What's your PXELinux config?

Steven.

4 Reply by stevenshiau

  • stevenshiau
  • Member
  • Offline
  • Registered: 2008-04-02
  • Posts: 293

Re: GParted webbooting

Oops... Now I found the problem here. I will try to fix that.

Steven.

5 Reply by dud225 (edited by dud225 2014-10-23 12:01:28)

  • dud225
  • Member
  • Offline
  • Registered: 2014-10-17
  • Posts: 12

Re: GParted webbooting

Thank you guys for your answesr. FYI my configuration is similar to what Cloudfront does : cloudboot.nchc.org.tw/cloudboot/BOOTMENU.php?boot=gparted-stable.

Could you please tell me what steps you are applying  to make memdisk booting sucessful ?

In the meanwhile I sent an email to Debian Live mailing list as, if I'm not misleading, Clonezilla & GParted are based upon : https://lists.debian.org/debian-live/20 … 00062.html in the hope that fixing it upstream would benefit to all  derived distros.

Hervé

6 Reply by stevenshiau

  • stevenshiau
  • Member
  • Offline
  • Registered: 2008-04-02
  • Posts: 293

Re: GParted webbooting

Actually I did nothing different for Clonezilla live. However, due to some reason the required modules are included in the initramfs. However, this is not the case for GParted live. I thought there was something triggers that (like the extra network modules in Clonezilla live cause update-initramfs to add them), but I can not find how.
In the end I force to add the two required modules, phram and mtdblock, in GParted live 0.20.0-1 (http://sourceforge.net/projects/gparted … /0.20.0-1/). However, new issue occurs, and the testing Clonezilla live I created at the same time has same issue:
Network interface NamePolicy= disabled on kernel commandline. Ignoring.

This seems to be related to systemd. However, I do not use systemd in Clonezilla/GParted live. They are forced to use sysvinit.
I will dig more. If you have patch about this, please let me know.

Steven.

7 Reply by stevenshiau

  • stevenshiau
  • Member
  • Offline
  • Registered: 2008-04-02
  • Posts: 293

Re: GParted webbooting

I uploaded GParted live 0.20.0-2, however, it still has issue so it won't  boot for your method 1.
I found the issue is actually on the program /usr/bin/memdiskfind.
Since syslinux version 5, this program fails to give the output. Thus it makes
modprobe phram phram=memdisk,$(/usr/bin/memdiskfind)
fail to load. You can see the error on system log.
I will report this issue to syslinux mailing.

Steven.

8 Reply by dud225

  • dud225
  • Member
  • Offline
  • Registered: 2014-10-17
  • Posts: 12

Re: GParted webbooting

stevenshiau wrote:

I uploaded GParted live 0.20.0-2, however, it still has issue so it won't  boot for your method 1.
I found the issue is actually on the program /usr/bin/memdiskfind.
Since syslinux version 5, this program fails to give the output. Thus it makes
modprobe phram phram=memdisk,$(/usr/bin/memdiskfind)
fail to load. You can see the error on system log.
I will report this issue to syslinux mailing.

Steven.

I'm testing with iPXE (from Qemu) and for now I'm not able to boot as well. I'm also getting the systemd error, however I'm noticing that I do not meet the same issue than you : memdiskfind is properly working and displaying memroy addresses. Unfortunately no device is created at boot. When retrying the same steps manually from the busybox environment it is working as expected : /dev/mtdblock0 comes up.
I did not dig too much for now but the memdisk wiki points out to ArchLinux implementation which is also adding udev rules : https://projects.archlinux.org/mkinitcp … isk.rules.

Hervé

9 Reply by stevenshiau

  • stevenshiau
  • Member
  • Offline
  • Registered: 2008-04-02
  • Posts: 293

Re: GParted webbooting

So the memdiskfind command works for qemu, however, it does not work on VMWare or a real machine I have (ASUSTeK BM6AE_BM1AE_BP1AE). For VMware running Ubuntu trusty x86 version, the error in /var/log/syslog when running command: modprobe phram phram=memdisk,/usr/bin/memdiskfind
==========================
[32475.751586] phram: not enough arguments
[32475.751594] do_init_module: 'phram'->init suspiciously returned 1, it should follow 0/-E convention
[32475.751594] do_init_module: loading module anyway...
[32475.751598] CPU: 0 PID: 2459 Comm: modprobe Not tainted 3.13.0-36-generic #63-Ubuntu
[32475.751600] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/31/2013
[32475.751602]  00000000 00000000 e6443e78 c1653137 f84e302c e6443f3c c10c43e6 c18348cc
[32475.751606]  c1670e72 f84e302c 00000001 c1670e72 00000000 e6443f94 f841f03c f84e302c
[32475.751610]  e67f99c8 f841e000 e67f99e4 00000001 00000001 f84e3068 00000000 e6443f60
[32475.751613] Call Trace:
[32475.751628]  [<c1653137>] dump_stack+0x41/0x52
[32475.751638]  [<c10c43e6>] load_module+0x1156/0x18e0
[32475.751645]  [<c10c4cd5>] SyS_finit_module+0x75/0xc0
[32475.751651]  [<c113a08b>] ? vm_mmap_pgoff+0x7b/0xa0
[32475.751660]  [<c16614cd>] sysenter_do_call+0x12/0x12
==========================

For real machine, same command I got:
==========================
[3912193.017380] phram: not enough arguments
[3912193.017383] do_init_module: 'phram'->init suspiciously returned 1,
n
[3912193.017383] do_init_module: loading module anyway...
[3912193.017386] CPU: 4 PID: 27372 Comm: modprobe Tainted: G           
.14.15-2~bpo70+1
[3912193.017387] Hardware name: ASUSTeK COMPUTER INC. BM6AE_BM1AE_BP1AE/BM6AE_BM1AE_BP1AE, BIOS 0902 10/17/2013
[3912193.017388]  0000000000000000 ffffffffa0b0e058 ffffffff814f8355 ffff8801841ffef8
[3912193.017390]  ffffffff810da52c ffffffff810d6b80 ffffc900075e4000 ffff880800000003
[3912193.017392]  ffffffff00000001 0000000000000000 ffffffffa0b0e270 ffff8801841ffe90
[3912193.017393] Call Trace:
[3912193.017398]  [<ffffffff814f8355>] ? dump_stack+0x41/0x51
[3912193.017401]  [<ffffffff810da52c>] ? load_module+0x1bcc/0x2540
[3912193.017403]  [<ffffffff810d6b80>] ? show_initstate+0x50/0x50
[3912193.017405]  [<ffffffff810daf6e>] ? SyS_init_module+0xce/0x110
[3912193.017408]  [<ffffffff8150812c>] ? sysenter_dispatch+0x7/0x21
==========================

I remember actually there are some limitations about using iso for PXE booting, e.g. if the RAM size is greater than 2 GB, it will fail. Besides, iso file could not be too large, either.
BTW, actually your method should be better. We use that all the time. It still works for GParted live 0.20.0-2 with the following PXE config:
==========================
label GParted-live
  # MENU DEFAULT
  # MENU HIDE
  MENU LABEL GParted Live 0.20.0-2-i686-pae (Ramdisk)
  # MENU PASSWD
  KERNEL GParted-live-vmlinuz
  APPEND initrd=GParted-live-initrd.img boot=live username=user config quiet noswap noeject  ip=  nosplash  noprompt fetch=tftp://$TFTP_SERVER/GParted-live-filesystem.squashfs

  TEXT HELP
  GParted Live 0.20.0-2-i686-pae runs on RAM
  ENDTEXT
==========================
You can get those files from GParted live zip file. What I did is just give them proper name.

Steven.

10 Reply by dud225

  • dud225
  • Member
  • Offline
  • Registered: 2014-10-17
  • Posts: 12

Re: GParted webbooting

stevenshiau wrote:

So the memdiskfind command works for qemu, however, it does not work on VMWare or a real machine I have (ASUSTeK BM6AE_BM1AE_BP1AE). For VMware running Ubuntu trusty x86 version, the error in /var/log/syslog when running command: modprobe phram phram=memdisk,/usr/bin/memdiskfind

Your command is incorrect, please try : modprobe phram phram=memdisk,$(/usr/bin/memdiskfind)

11 Reply by dud225 (edited by dud225 2014-10-25 14:51:55)

  • dud225
  • Member
  • Offline
  • Registered: 2014-10-17
  • Posts: 12

Re: GParted webbooting

Hello Steven

I've made some interesting progress about that issue. I noticed that after having fallen back on Busybox because no root fs could be found, the boot.log file still affirms that the memdisk stuff worked as expected :
https://lut.im/MxFV1oZl/ff3faXkC

When executing commands "modprobe phram phram=memdisk,$(/usr/bin/memdiskfind) && modprobe mtdblock" I have no error but there is no mtdblock device showing up either. After having unloaded both modules I came successful : "modprobe -r phram mtdblock && . lib/live/boot.sh && Main" mounted rootfs under /root.

As a conclusion memdisk booting is failing because phram & mtdblock modules are already loaded without the right parameters. Forcing the addition of both modules presumably in file conf/modules makes them being incorrectly loaded right after the boot and that explains why using the same build procedure for Clonezilla makes it fail as well.

Hervé

12 Reply by stevenshiau

  • stevenshiau
  • Member
  • Offline
  • Registered: 2008-04-02
  • Posts: 293

Re: GParted webbooting

The problem I have here is, no matter which machine I run with /usr/bin/memdiskfind (from syslinux-6.03), on Debian Wheezy or Ubuntu Trusty, they all gave me nothing:
root@trusty:~# /usr/bin/memdiskfind
root@trusty:~#
Therefore here all the tests I had about:
modprobe phram phram=memdisk,$(/usr/bin/memdiskfind)
definitely failed.
Could you try to run "/usr/bin/memdiskfind" on some of your real machines?
Does it give you the memory start and length numbers?

Steven.

13 Reply by dud225 (edited by dud225 2014-10-25 22:52:08)

  • dud225
  • Member
  • Offline
  • Registered: 2014-10-17
  • Posts: 12

Re: GParted webbooting

Yes I get the same result :
hwerner@PC-hwerner:~$ syslinux --version
syslinux 6.03  Copyright 1994-2014 H. Peter Anvin et al
hwerner@PC-hwerner:~$ sudo memdiskfind
hwerner@PC-hwerner:~$
hwerner@PC-hwerner:/tmp$ echo $?
1


but this isn't surprising, this tool only works from a memdisk environment :

The manpage wrote:

The memdiskfind utility searches memory for a memdisk instance, and, if found, outputs the parameters needed to use the [p]hram driver in Linux to map it

Please boot Gparted from memdisk. Please try with Qemu or use this ISO : http://boot.ipxe.org/ipxe.iso and type in CTRL+B to enter iPXE shell and then write the following commands :

dhcp
initrd http://downloads.sourceforge.net/project/gparted/gparted-live-testing/0.20.0-2/gparted-live-0.20.0-2-amd64.iso
chain http://boot.salstar.sk/memdisk iso raw ||
prompt

Let me know if you need more information.

Hervé

14 Reply by stevenshiau

  • stevenshiau
  • Member
  • Offline
  • Registered: 2008-04-02
  • Posts: 293

Re: GParted webbooting

OK, now I can reproduce your problem. For the tests I had before, I used gparted-live-0.20.0-2-i686-pae.iso, that's why I got the different results.
This is really weird, though.
I turned on the verbose mode in /lib/live/boot.sh by adding "set -x", then remastered an iso. With this, apparently I can see the phram module was loaded with correct command and no error message:

+ [ -x /usr/bin/memdiskfind ]
+ /usr/bin/memdiskfind
+ MEMDISK=0x32478000,0xda68000
+ [ 0 -eq 0 ]
+ modprobe phram phram=memdisk,0x32478000,0xda68000
+ modprobe mtdblock

No idea why /dev/mtd* were not created, and as you mentioned, if phram module was unloaded then reloaded with same command, the /dev/mtd* files were generated. Not sure if this is related to udev or something else. It used to worked very well, not only amd64 iso, but also i686-pae iso. Now the i686-pae one has worse result. Very weird...

Steven.

15 Reply by dud225 (edited by dud225 2014-10-26 11:32:41)

  • dud225
  • Member
  • Offline
  • Registered: 2014-10-17
  • Posts: 12

Re: GParted webbooting

stevenshiau wrote:

Not sure if this is related to udev or something else

Could you try to add the udev rule used by ArchLinux to see if it triggers any change :

ACTION=="add|change", SUBSYSTEM=="block", KERNEL=="mtdblock*", ATTRS{name}=="memdisk", ATTRS{type}=="ram", GOTO="memdisk_start"
GOTO="memdisk_end"

LABEL="memdisk_start"

SYMLINK+="memdisk"
IMPORT{builtin}="blkid"
ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"
ENV{ID_FS_USAGE}=="filesystem|other", ENV{ID_FS_LABEL_ENC}=="?*", SYMLINK+="disk/by-label/$env{ID_FS_LABEL_ENC}"

LABEL="memdisk_end"

EDIT :

http://reboot.pro/topic/11690-memdisk-driver-for-linux/ wrote:

This udev rules creates a /dev/memdisk symlink and the usual /dev/disk/by-{uuid,label} symlinks, as udev's default rules ignore mtd*

16 Reply by stevenshiau

  • stevenshiau
  • Member
  • Offline
  • Registered: 2008-04-02
  • Posts: 293

Re: GParted webbooting

I added a workaround to live-boot so now the AMD64 version of GParted live (gparted-live-0.20.0-3-amd64.iso) should work. However, same method does not work for i486 or i686-pae versions. I think it's kernel-related issue.

Steven.

17 Reply by dud225

  • dud225
  • Member
  • Offline
  • Registered: 2014-10-17
  • Posts: 12

Re: GParted webbooting

Great I confirm amd64 build is successfully booting through memdisk. However I noticed that you added a quick fix in boot scripts :

                                        while [ "$i" -lt 10 -a ! -e /dev/mtdblock0 ]; do
                                                # Try 10 times to get /dev/mtdblock0. This is a workaround due to some weird case that the 1st phram module loading, no /dev/mtdblock0 is created.
                                                # Ref: http://gparted-forum.surf4.info/viewtopic.php?id=17263
                                                rmmod mtdblock > /dev/null 2>&1
                                                rmmod phram > /dev/null 2>&1
                                                # We found a memdisk, set up phram
                                                modprobe phram phram=memdisk,${MEMDISK}

                                                # Load mtdblock, the memdisk will be /dev/mtdblock0
                                                modprobe mtdblock
                                                i="$(($i + 1))"
                                                sleep 0.1
                                        done

Did you try the udev rule I gave you in my last post ?

Concerning the 32 bits build, this doesn't work on Qemu but Clonezilla 32 bits isn't working either (through memdisk).

Hervé

18 Reply by stevenshiau

  • stevenshiau
  • Member
  • Offline
  • Registered: 2008-04-02
  • Posts: 293

Re: GParted webbooting

dud225 wrote:

Did you try the udev rule I gave you in my last post ?

Yes, I tried that. Not working though.
Therefore that's why I think this is kernel related issue.

Steven.

19 Reply by dud225

  • dud225
  • Member
  • Offline
  • Registered: 2014-10-17
  • Posts: 12

Re: GParted webbooting

stevenshiau wrote:

Yes, I tried that. Not working though.
Therefore that's why I think this is kernel related issue.
Steven.

OK thanks for testing. This is strange since Gparted is pretty close to Clonezilla isn't it ?

Could you tell me when this change will be applied in the stable release ? Do you think it needs more testing ?

20 Reply by stevenshiau

  • stevenshiau
  • Member
  • Offline
  • Registered: 2008-04-02
  • Posts: 293

Re: GParted webbooting

Yes, this is weird. However, a friend Ady sent me an email suggesting to add "vmalloc=256MiB" in the boot parameters. After trying that, I confirmed it's working for i686-pae versions of 0.20.0-3.
Therefore this also explains that why in the past Clonezilla live worked, but GParted live did not.
The iso file of Clonezilla live iso was less than 128 MB in the past. GParted live iso was always larger than 128 MB. Default vmalloc for Linux kernel on i486/i686 is 128 MB.
I still need to know if "vmalloc=256MiB" is added to the default boot parameters, any impacts... like to cause the low memory machine (~ 512 MB RAM) not able to boot GParted live.

Steven.

21 Reply by stevenshiau

  • stevenshiau
  • Member
  • Offline
  • Registered: 2008-04-02
  • Posts: 293

Re: GParted webbooting

BTW, I have started using live-boot 3.0.1-1.drbl5, so from GParted live 0.20.0-3, the amd64 version of GParted live should work. As the timing for stable release, I think if no bad influences of adding "vmalloc" boot parameter, I will create another version, and if the project leader, Curtis, agrees, it will be moved to stable one.
For the time being, please use gparted-live-0.20.0-3-i686-pae.iso.

Steven.

22 Reply by dud225

  • dud225
  • Member
  • Offline
  • Registered: 2014-10-17
  • Posts: 12

Re: GParted webbooting

stevenshiau wrote:

Yes, this is weird. However, a friend Ady sent me an email suggesting to add "vmalloc=256MiB" in the boot parameters. After trying that, I confirmed it's working for i686-pae versions of 0.20.0-3.
Therefore this also explains that why in the past Clonezilla live worked, but GParted live did not.
The iso file of Clonezilla live iso was less than 128 MB in the past. GParted live iso was always larger than 128 MB. Default vmalloc for Linux kernel on i486/i686 is 128 MB.
I still need to know if "vmalloc=256MiB" is added to the default boot parameters, any impacts... like to cause the low memory machine (~ 512 MB RAM) not able to boot GParted live.

Thanks for your explanation.
Well done Ady smile

stevenshiau wrote:

I have started using live-boot 3.0.1-1.drbl5, so from GParted live 0.20.0-3, the amd64 version of GParted live should work [....] For the time being, please use gparted-live-0.20.0-3-i686-pae.iso.

I'm sorry, I don't get what you mean ? I can successfully boot version 0.20.0-3 amd64 yet.

23 Reply by stevenshiau

  • stevenshiau
  • Member
  • Offline
  • Registered: 2008-04-02
  • Posts: 293

Re: GParted webbooting

dud225 wrote:

I'm sorry, I don't get what you mean ? I can successfully boot version 0.20.0-3 amd64 yet.

I meant you do not have to care it's stable release or not. Just remember those versions >= 0.20.0-3 amd64 release should work.

Steven.

24 Reply by gedakc

  • gedakc
  • Developer
  • Offline
  • Registered: 2007-11-30
  • Posts: 4,593

Re: GParted webbooting

Hi Steven,

I can test another live version to ensure that it boots on my test computers.  Assuming that passes, then we can promote the live image to stable.


dud225 wrote:

I can successfully boot version 0.20.0-3 amd64 yet.

dud225, are you saying that you can or cannot boot the 0.20.0-3 amd6 live image?

25 Reply by dud225

  • dud225
  • Member
  • Offline
  • Registered: 2014-10-17
  • Posts: 12

Re: GParted webbooting

gedakc wrote:

dud225, are you saying that you can or cannot boot the 0.20.0-3 amd6 live image?

Booting 0.20.0-3 amd64 through memdisk is working properly on my computer.
I did not try i686 build however.