[ipxe-devel] Does imgextract keep writing to same named location by design?

[John Hanks]

Also forgot to answer part of that, yes these are all gzip compressed cpio
archives, I'm using imgextract because the kernel won't uncompress an
initrd larger than 4 GB.

griznog


On Fri, Mar 11, 2022 at 1:55 PM John Hanks <griznog at gmail.com> wrote:

> Whether contents from all of them are available turns out to be hard to
> answer. Some background for what I'm doing, I have a container (basically a
> chroot OS install) image that is my golden img. Then I have additional
> smaller images, each one containing some configuration subset that I want
> to overlay onto the golden OS image to modify it. So I can't easily use
> size to say it's doing the right thing because I'm overwriting (or
> attempting to) some files and I'd expect the result to not be the simple
> sum of the parts.
>
> The best example of observed behavior is my ssh config setup. The golden
> image contains the keys generated when the initial chroot install was done.
> Then I have an ssh img that overlays that and places the correct keys for
> the host being booted over those. The overlay ssh image also adds several
> additional files in /etc/ssh.
>
> If I get the OS container first, then get the ssh overlay, then the files
> that are new from the overlay appear correctly, but files that should
> overwrite those in the container don't appear so the new keys aren't there.
>
> If I get the overlay img first, then the container, the files from the
> overlay all get written correctly and the ones in the container do not
> overwrite them.
>
> Basically it behaves as if the first extracted img to write a file wins
> for that file and all subsequent files with the same name are discarded.
> This is counter-intuitive as I would have expected the last writer to win
> if I had simply ran `cpio` to extract all the images to a single location.
>
> Other than that bit of confusion, I haven't yet found an example of
> anything missing that should be there in the resulting booted host
> filesystem.
>
> Despite it being confusing with first writer winning, it's still quite
> useful behavior as collapsing all these overlay images before passing them
> to the kernel simplifies the resulting kernel command line nicely.  Is it
> worth adding a feature request for this if it's really unintentional? It's
> trivial to use different names to keep them separate so if the merging
> behavior was documented and stable, it'd be a nice option to have
> available, especially if it could behave predictably like extracting a
> series of cpio archives to the same location.
>
> griznog
>
>
> On Fri, Mar 11, 2022 at 12:54 PM Christian Iwata Nilsson <nikize at gmail.com>
> wrote:
>
>> On Fri, 11 Mar 2022 at 17:42, John Hanks <griznog at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I am using imgextract to download several images, like so
>>>
>>> imgextract --name one http://master/one.img
>>> imgextract --name two http://master/two.img
>>> imgextract --name three http://master/three.img
>>> boot kernel initrd=initrd.magic initrd=one initrd=two initrd=three blah
>>> blah blah
>>>
>>> So far so good, except that I wanted to try adding a lot of these but am
>>> restricted by the maximum kernel command line length. I naively tried
>>>
>>> imgextract --name mysystem http://master/one.img
>>> imgextract --name mysystem http://master/two.img
>>> imgextract --name mysystem http://master/three.img
>>> boot kernel initrd=initrd.magic initrd=mysystem blah blah blah
>>>
>>> and it worked!!! But, that seems way too easy, like I'm cheating. Is
>>> this expected to work this way or am I exploiting a bug that might get
>>> fixed later? I don't want to depend on this unless it's working this way on
>>> purpose.
>>>
>>> Thanks,
>>>
>>> griznog
>>>
>>>
>> Are you sure that the contents from them are all available?
>> Since you are using imgextract I assume that the contents of *.img is
>> compressed and that it contains CPIO archives?
>>
>> Could you verify with imgstat that size adds up if you do them to
>> separate names and then try those 2 to the same file?
>> I see nothing that indicates this as being expected behaviour.
>>
>> /Christian
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ipxe.org/pipermail/ipxe-devel/attachments/20220311/e8cbeecd/attachment.htm>