FS#13213 - use isohybrid to make iso that can boot on usb also

Attached to Project: Release Engineering
Opened by christopher rogers (godane) - Wednesday, 11 February 2009, 20:07 GMT
Last edited by Dieter Plaetinck (Dieter_be) - Friday, 05 March 2010, 18:12 GMT
Task Type Feature Request
Category ArchISO
Status Closed
Assigned To Aaron Griffin (phrakture)
Gerhard Brauer (GerBra)
Dieter Plaetinck (Dieter_be)
Thayer Williams (thayer)
Architecture All
Severity Medium
Priority Normal
Reported Version 2009.02-RC
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 3
Private No

Details

Isohybrid can allow you to make a iso that can be put on usb with dd or cat. This will save you on making 2 images and save on bandwidth.

http://syslinux.zytor.com/wiki/index.php/Doc/isolinux#HYBRID_CD-ROM.2FHARD_DISK_MODE

isohybrid archiso.iso

Thats all you have to do for usb support.
This task depends upon

Closed by  Dieter Plaetinck (Dieter_be)
Friday, 05 March 2010, 18:12 GMT
Reason for closing:  Implemented
Additional comments about closing:  http://projects.archlinux.org/archiso.gi t/commit/?id=ca52931e3aa7b948fb1957240c8 80770b71d0e6b

implemented in archiso git.
archiso now builds iso/usb hybrid images.
Comment by christopher rogers (godane) - Wednesday, 11 February 2009, 23:20 GMT
I fould out it doesn't work with klibc mount. It mounts fine on archlinux system. I just don't think it works with klibc mount. :(
Comment by christopher rogers (godane) - Thursday, 12 February 2009, 00:15 GMT
Looks like i made a mistake. The file that was need in ordet to find the cd was not there. Iso with isohybrid does work fine.
Comment by Aaron Griffin (phrakture) - Friday, 13 February 2009, 16:47 GMT
Hmm that can be fixed. I can play around with getting isohybrid working.
Comment by Greg (dolby) - Thursday, 01 October 2009, 18:26 GMT
For this to be implemented archiso has to switch to syslinux. archboot already does that, and uses isohybrid.
Comment by Aaron Griffin (phrakture) - Thursday, 01 October 2009, 19:24 GMT
Yeah, I forgot to mention that part - isohybrid requires syslinux
Comment by Sven-Hendrik Haase (Svenstaro) - Wednesday, 28 October 2009, 17:06 GMT
I don't see what's bad about syslinux though. I just did a successful test on my own Arch Linux-based distro using isohybrid and it worked really quite well. I think it can be argued that syslinux is just as comaptible (if not better) as grub but with isohybrid it has a great advantage over grub. Using the current archiso master as a base, it is a trivial task to use syslinux + isohybrid. I know you (Aaron) don't like syslinux but I think you should get over that in favor of hybrid images.
Comment by Aaron Griffin (phrakture) - Wednesday, 28 October 2009, 20:48 GMT
Yes, I've been thinking along that route. While grub is nice and all, reverting to syslinux might make the whole distribution process easier. For anyone interested, perhaps providing an alternate configuration (in the archiso git tree) can help speed this along.

Comment by Sven-Hendrik Haase (Svenstaro) - Thursday, 29 October 2009, 12:41 GMT
No problem, I will send you my current (working) archiso fork that does syslinux and isohybrid just fine once you merge the 2nd patch I sent you for archiso last week (because else it gets messy for me to keep track of patches).
Comment by Aaron Griffin (phrakture) - Thursday, 29 October 2009, 14:00 GMT
No problem. I have about 10 pending archiso patches to apply :S
Comment by Sven-Hendrik Haase (Svenstaro) - Saturday, 14 November 2009, 01:29 GMT
Meh Aaron come on now, I want to get isohybrid support in.
Comment by Aaron Griffin (phrakture) - Tuesday, 17 November 2009, 23:33 GMT
http://mailman.archlinux.org/pipermail/arch-dev-public/2009-November/014221.html

I've been out of town. I'll get to it, but keep in mind I *do* have a life and *do* do other things. I cannot always be on everyone's schedules
Comment by Aaron Griffin (phrakture) - Monday, 23 November 2009, 20:55 GMT
Patch is merged and pushed.

If you have a remote git tree somewhere I could follow, that would be very helpful
Comment by Sven-Hendrik Haase (Svenstaro) - Tuesday, 24 November 2009, 21:54 GMT
Nice. I made a public repo: http://github.com/Svenstaro/archiso-lglive

How to go about the isohybrid changes? I was going to kick out all the deprecated grub and traditional usb-hacking stuff because it keeps the codebase small nice and it probably won't be needed anyway as isohybrids are surprisingly compatible. I was also going to integrate the isohybrid stuff to mkarchiso instead of the Makefiles.
Comment by Aaron Griffin (phrakture) - Tuesday, 24 November 2009, 22:32 GMT
Hmm, adding Gerhard and Dieter on here.

Considering how much cleaner it would make everything, I am all for just getting rid of grub for the ISOs.

We can worry about making syslinux pretty later. The only thing we lose is the actual ability to boot an existing system (right?)

Comment by Sven-Hendrik Haase (Svenstaro) - Wednesday, 25 November 2009, 00:01 GMT
About booting an existing Linux:
DEFAULT linux
LABEL linux
SAY Now booting the kernel from SYSLINUX...
KERNEL vmlinuz.img
APPEND ro root=/dev/sda1 initrd=initrd.img
Straight from the syslinux wiki.
Of course, this seems quite inconvenient because you have to know everything exactly.
However, if you use chain.c32 (which is already included for us but we need to copy it) then you can also
chainboot grub or basically any other installed bootloader. So no, we don't lose anything. We even have interactive editing back thanks to vesamenu.c32.
Comment by Sven-Hendrik Haase (Svenstaro) - Wednesday, 25 November 2009, 00:17 GMT
Come to think of it, do we still need two different initcpios? pata/ide?
Comment by Aaron Griffin (phrakture) - Wednesday, 25 November 2009, 00:20 GMT
With vesamenu.c32 we can edit the above line just like in grub? That's more what I was indicating (not everyone uses /dev/sda3 for their root). Can chain.c32 chain to a bootloader on the ISO as well (say, if we left grub on there)?

This is sounding more and more promising. I'm now following your changes here: http://code.phraktured.net/cgit.cgi/archiso/

As for the actual changes. Perhaps it might be idea just to copy the install-iso config dir and start fresh. That was the original intent of that - to maintain multiple configurations that might/should actually work
Comment by Aaron Griffin (phrakture) - Wednesday, 25 November 2009, 00:21 GMT
I think Thomas indicated that we do not need the legacy initcpio stuff anymore. I am not 100% sure on that, though
Comment by Sven-Hendrik Haase (Svenstaro) - Wednesday, 25 November 2009, 00:31 GMT
Very well, I'll just delete crap and trim it down as good as possible. As for what chain.c32 can do: http://syslinux.zytor.com/wiki/index.php/Comboot/chain.c32. Apart from that I'm not sure how self-chaining an iso will work but there most probably is some kind of adddon .c32 for that as well. One thing that might work is getting a grub compiled to a single .bin (those exist!) and supply that to the syslinux kernel line. This shouldn't be a priority though and it would be rather silly too.

Let's get familar with syslinux and screw the rest for now. I've been using it for my own distro for a while now and it is a lot cooler than what it looks like on the surface. My patch got merged rather quick as well so development is going strong. You'll see when I have some results.
Comment by Aaron Griffin (phrakture) - Wednesday, 25 November 2009, 18:44 GMT
I made a dummy untested commit to my repo here:
http://code.phraktured.net/cgit.cgi/archiso/log/?h=isohybrid
Comment by Sven-Hendrik Haase (Svenstaro) - Wednesday, 25 November 2009, 18:50 GMT
Ah no good, I'm working on own of my own right now. Let's not duplicate work. I didn't push yet, hang on a little.
Comment by Aaron Griffin (phrakture) - Wednesday, 25 November 2009, 19:11 GMT
It was about 2 minutes worth of work while I was waiting for a compile here at work. Not really duplication, I just wanted to see what could be cleaned up.

Also, please use the latest makefile if you're planning on copying that over, as it does things much better
Comment by Sven-Hendrik Haase (Svenstaro) - Wednesday, 25 November 2009, 21:20 GMT
Your git repo is rather strange. I can't clone it or check it out at all.
git clone -b isohybrid http://code.phraktured.net/git/arch/archiso/.git archiso-phrak
Initialized empty Git repository in /home/svenstaro/archiso-phrak/.git/
warning: Remote branch isohybrid not found in upstream origin, using HEAD instead
warning: remote HEAD refers to nonexistent ref, unable to checkout.

Neither can just git clone without -b do it. What is wrong here?
Comment by Aaron Griffin (phrakture) - Wednesday, 25 November 2009, 21:24 GMT
Guess I don't have the automatic git update-server-info on there. It's all fixed now
Comment by Sven-Hendrik Haase (Svenstaro) - Wednesday, 25 November 2009, 22:59 GMT
Check out my repo now. It's working totally awesome and looks nice too. isohybrid is working nicely as well and I tested it in a physical machine using both USB and CD. It's almost mission accomplished. Also, I got rid of a lot of stuff in mkarchiso as well.
Comment by Aaron Griffin (phrakture) - Wednesday, 25 November 2009, 23:08 GMT
So... two questions now:

a) Do we kill off grub support from mkarchiso entirely? I'd like Gerhard or Dieter to weigh in on this. One can always install grub manually and skip the "iso" step to manually use another bootloader.

b) Screenshots of the menu please?!
Comment by Aaron Griffin (phrakture) - Wednesday, 25 November 2009, 23:12 GMT
Ok, this weekend is Thanksgiving here in the US, so I'll be visiting family until Friday. Don't expect too much movement on this until Saturday or so.

The real question now is: do we completely kill off grub support? I'd like other devs to weigh in. My current vote is 'yes'
Comment by Sven-Hendrik Haase (Svenstaro) - Wednesday, 25 November 2009, 23:38 GMT
I have no hard feelings at all killing off grub. Especially considering how old and patched up it is right now. syslinux in comparison is still actively developed and widely deployed. It removes a lot of clutter from our scripts.

Awesome image is here: http://imgur.com/GbvHo
By the way, I included a script "test-isolinux.sh" which you can call if you have qemu-kvm, cdrkit and syslinux installed and it will let you rapidly test your isolinux settings.

Aaron, take pictures from your family trip and make the best one the next isolinux splash graphic.
Comment by Sven-Hendrik Haase (Svenstaro) - Friday, 27 November 2009, 19:00 GMT
I now also merged the patch against the silly pwgen stuff. I don't see much wrong with my trunk right now. Stuff that still needs to be done: hard disk booting, release notes, help, that's it.
Comment by Dieter Plaetinck (Dieter_be) - Sunday, 29 November 2009, 10:18 GMT
let's see. users can use grub on our current media for: booting two live environment flavors (normal and legacy), reading release notes, run memtest and x86test, lilo diagnostic (useful?), reinstall grub immediately, play space invaders (useful?), boot existing linux/windows/nfs root/pxe, shutdown and/or reboot computer, custom boot (edit commandline).

what would we miss with isolinux?
booting other OS's doesn't seem to work as well (http://syslinux.zytor.com/wiki/index.php/ISOLINUX#Can_I_boot_other_operating_systems_with_ISOLINUX.3F), and i'm not sure if you can boot from nfs/pxe with it.
but other then that, we're good right? did anyone try chainloading from syslinux to grub? if we would support such as chainload to grub, does that mean we still need all our messy code (i.e. not really an improvement?)
the chainload seems a great alternative if we can keep all our nice code cleanups, but if isolinux doesn't really lack anything too important i would not even provide chainloading grub.
Comment by Sven-Hendrik Haase (Svenstaro) - Sunday, 29 November 2009, 21:25 GMT
We can probably chainload other bootloaders using chain.c32. If we want to use grub, we can put it onto a memdisk and let isolinux load that. However, we should also ask what is useful and what is not. Doing localboots to existing Linux and Windows installations probably is useful. How useful is nfs/pxe booting when every modern BIOS does PXE anyway?
Comment by Gerardo Exequiel Pozzi (djgera) - Sunday, 29 November 2009, 22:48 GMT
> Come to think of it, do we still need two different initcpios? pata/ide?
Not sure, at least in udev git [#1] all rules related to "ide" are removed. (for future udev-148)

[#1] http://git.kernel.org/?p=linux/hotplug/udev.git;a=commitdiff;h=ecf61aa7fdb5b7e9e19f06584caee9b5eb1c7350
Comment by Aaron Griffin (phrakture) - Monday, 30 November 2009, 21:12 GMT
Sorry for the delay in merging this. I am very interested in this, so am going to look into it a bit right now.

I'm going to cherry-pick some patches and then merge your syslinux config into the syslinux-iso tree I created. The intent is to keep both configs side-by-side until we feel like completely killing the other one off
Comment by Dan McGee (toofishes) - Monday, 30 November 2009, 21:34 GMT
A huge +1 on cutting back on the number of images we have to ship. I don't know how much I can add to development time here but I can try to do some testing once we get some working images.
Comment by Aaron Griffin (phrakture) - Monday, 30 November 2009, 22:42 GMT
Please check master here:
http://code.phraktured.net/cgit.cgi/archiso/

Sven, I merged in your archiso changes and then copied the entire syslinux config to the directory I originally created. This way, removal of other configs doesn't need to be done as part of this and we can leave the others in place.

Please take a look. I also created a net image (make net-iso) which should be complete in a few minutes:
http://phraktured.net/archlinux-2009.12-netinstall-x86_64.iso
Comment by Dan McGee (toofishes) - Monday, 30 November 2009, 23:07 GMT
You have one patch in there that has a busted author, if you care (d924b40051db1). git rebase -i, git commit --amend --author "xxx", git rebase --continue can be your friend.
Comment by Aaron Griffin (phrakture) - Monday, 30 November 2009, 23:14 GMT
Fixed, thanks for noticing
Comment by Sven-Hendrik Haase (Svenstaro) - Tuesday, 01 December 2009, 16:09 GMT
Aaron, your image looks good and boots fine here. So unless anybody is utterly disgruntled now by removing grub, I suggest we close this bug now (as we now do create isohybrid images by default) and be glad that we will only have to offer two images for download for now as opposed to six as before. A few changes for the releasechanges.txt, the automatic isolinux menu naming using sed (the part where I put in XXX YYY at the menu), the contextual isolinux menu help and we should be able to ask people on the mailing list to test these new images. Or is there anything I overlooked?
Comment by Aaron Griffin (phrakture) - Tuesday, 01 December 2009, 16:58 GMT
I say we leave this open just for a bit. I really want to crack away at some of this stuff and was hoping for input from Dieter and/or Gerhard here. I'd like to use this bug for additional work to get things like help in there

Also tagged Thayer to get a good background image and menu color scheme (see screenshot a few posts up - I snagged that from a random google result for "arch linux wallpaper"). It can be PNG now :)
Comment by Sven-Hendrik Haase (Svenstaro) - Tuesday, 01 December 2009, 19:01 GMT
Very well. Thayer you can use the test script I put in there to rapidly test the menu and its colors. We can all agree though that we will be using this whole isolinux stuff for the next release of the installation media and for the future so our efforts have not gone to waste? ;)
Comment by Aaron Griffin (phrakture) - Tuesday, 01 December 2009, 19:13 GMT
Considering Dan, Dieter, and myself like this (primarily due to simplification of the whole process), then yes.

I would like to leave this open simply to make sure the new isolinux image does not lose anything from the grub images that was useful.
Comment by Gerardo Exequiel Pozzi (djgera) - Wednesday, 02 December 2009, 05:44 GMT
Question: this parameter (radeon.modeset=0) at kernel command line on new isolinux.cfg is intentional or introduced by error?
Comment by Thayer Williams (thayer) - Wednesday, 02 December 2009, 05:57 GMT
I'm not sure what our timeframe is for this, but I'm swamped with final projects and exams until Dec 22, and then of course it's Christmas. After Boxing Day (Dec 26 for you yankees) my schedule opens wide up for dev stuff.
Comment by Sven-Hendrik Haase (Svenstaro) - Wednesday, 02 December 2009, 15:59 GMT
radeon.modeset=0 is there because I noticed that my laptop (radeon mobility 9000 aka rev 250) would not boot a kernel 2.6.31 from a live cd without that. I suggest we leave it in until KMS behaves more stable. Thayer that will be alright, I'll take over the image and nice looking stuff. Do we want it dark or light (like archlinux.org)?
Comment by Aaron Griffin (phrakture) - Wednesday, 02 December 2009, 17:14 GMT
Hmm, can we see screenshots of both dark and light options? Light might be neat if it works right for the 16 colors we're limited too.

Regarding KMS, that one is iffy. At the very least, this should wait until the next kernel release when KMS is (hopefully) stabler, but for now setting KMS to off gives us maximum bootability
Comment by Dieter Plaetinck (Dieter_be) - Wednesday, 02 December 2009, 19:38 GMT
So I was pretty sure there was a reason we said "no isolinux-only" the previous releases, and I only found:
- some uncertainity whether isolinux would work with USB sticks (not an issue anymore apparently)
- Aarons comments @ http://mailman.archlinux.org/pipermail/arch-releng/2009-February/000258.html which are hopefully outdated as wel.
Comment by Aaron Griffin (phrakture) - Wednesday, 02 December 2009, 19:55 GMT
Yeah. I still think isolinux is kinda crappy (technology-wise), but I was unaware of all the flexibility of the c32 stuff that now exists. So the inflexible part is moot now
Comment by Sven-Hendrik Haase (Svenstaro) - Friday, 04 December 2009, 04:05 GMT
Screenshots. Dark: http://imgur.com/28ecK&XwxYe Light: http://imgur.com/28ecK&XwxYel
With a few minor adjustments, I think both look reasonable. Opinions?
Comment by Gerardo Exequiel Pozzi (djgera) - Friday, 04 December 2009, 14:16 GMT
+1 for dark, anyway Light looks good (+0.9) ;)

About remoing the old IDE initrd: Please see [#1] seems that is necessary for now.

[#1] http://mailman.archlinux.org/pipermail/arch-dev-public/2009-December/014521.html
Comment by Sven-Hendrik Haase (Svenstaro) - Friday, 04 December 2009, 14:50 GMT
Ah crap. Our Makefile just started to get uncomplicated.
Comment by Sven-Hendrik Haase (Svenstaro) - Saturday, 05 December 2009, 14:12 GMT
So any more opinions on the colors? I'm thinking I should maybe ask the mailing list.
Comment by Dan McGee (toofishes) - Saturday, 05 December 2009, 14:52 GMT
+1 for dark.
Comment by Dieter Plaetinck (Dieter_be) - Saturday, 05 December 2009, 14:57 GMT
i usually prefer light themes but in this case dark really looks better. (+it consumes less energy)
Comment by Thayer Williams (thayer) - Saturday, 05 December 2009, 20:44 GMT
I agree with Dieter, the dark looks sharp in this case.
Comment by Sven-Hendrik Haase (Svenstaro) - Tuesday, 08 December 2009, 00:06 GMT
I cleaned up my branch quite a lot (http://github.com/svenstaro/archiso-lglive). I think we're ready for two test images for the mailing list, are we not? We don't have to wait for kernel 2.6.32 to get out of testing because it appears to be quite buggy and KMS got even worse. Thoughts?
Comment by Gerardo Exequiel Pozzi (djgera) - Tuesday, 08 December 2009, 12:46 GMT
Good!, but the initial motivation for next archiso installation media is just to include the new kernel [#1]. We also need the new initscript-2009.11 (in testing) that fixes this: " FS#16497  - [initscripts] rc.sysinit and /etc/mtab small issue with mount.aufs" or just put the current rc.sysinit on overlay fs.
There are small pending changes in my branch since last merge in master [#2]

[#1] http://mailman.archlinux.org/pipermail/arch-releng/2009-November/000792.html
[#2] http://github.com/djgera/archiso/commits/djgera
Comment by Sven-Hendrik Haase (Svenstaro) - Tuesday, 08 December 2009, 18:17 GMT
Oh I see, thanks for the information. We will have some time to polish the new stuff then as kernel 2.6.32 appears to be rather buggy, especially with KMS.
Comment by Sven-Hendrik Haase (Svenstaro) - Wednesday, 30 December 2009, 01:24 GMT
Can we resolve this finally? I want to update the wiki on this but I can't do it in the twilight of grub and isolinux. Let's make it just isolinux and call it a day and close this bug.
Comment by Dieter Plaetinck (Dieter_be) - Thursday, 31 December 2009, 14:47 GMT
fine by me. but what do you mean with "want to update the wiki" ? which wiki? shouldn't we actually start building images first? ;)
Comment by Sven-Hendrik Haase (Svenstaro) - Thursday, 31 December 2009, 21:04 GMT
Our wiki: http://wiki.archlinux.org/index.php/Archiso
Many people use that article as a reference for archiso and as of the current git version it is somewhat out of date.
Comment by Markus M. May (triplem) - Friday, 01 January 2010, 18:15 GMT
Right now, the biggest problem seems to be that the Git package contains wrong configs, with which you cannot build any ISO. Took me quite some time to figure this out :-(
Comment by Sven-Hendrik Haase (Svenstaro) - Saturday, 02 January 2010, 12:39 GMT
Indeed, Aaron left all non-syslinux configs in there as well because he considered we might be going back to grub. I think the old configs should be deleted or moved in order to avoid confusion.
Comment by Markus M. May (triplem) - Saturday, 02 January 2010, 13:28 GMT
Creating a Branch would serve the purpose, I believe. But now the trunk is a little screwed anyway. I think that you (Sven) should create a tag/branch and the merge your own repo into here.
Comment by Sven-Hendrik Haase (Svenstaro) - Saturday, 02 January 2010, 13:49 GMT
I do have a branch for this purpose: http://github.com/svenstaro/archiso-lglive
I do however not possess rights for Aarons repo and he won't give me any ( :( ) and thus all I can do is hope for him to merge my stuff. Personally, I thought it was a good idea to remove all the broken configs. My repo is currently up-to-date and merges nicely.
Comment by Markus M. May (triplem) - Saturday, 02 January 2010, 22:51 GMT
Creating a Branch would serve the purpose, I believe. But now the trunk is a little screwed anyway. I think that you (Sven) should create a tag/branch and the merge your own repo into here.
Comment by Markus M. May (triplem) - Saturday, 02 January 2010, 22:53 GMT
Sorry for the dup. My fault.

Loading...