[ipxe-devel] Does imgextract keep writing to same named location by design?

[John Hanks]

Whether contents from all of them are available turns out to be hard to
answer. Some background for what I'm doing, I have a container (basically a
chroot OS install) image that is my golden img. Then I have additional
smaller images, each one containing some configuration subset that I want
to overlay onto the golden OS image to modify it. So I can't easily use
size to say it's doing the right thing because I'm overwriting (or
attempting to) some files and I'd expect the result to not be the simple
sum of the parts.

The best example of observed behavior is my ssh config setup. The golden
image contains the keys generated when the initial chroot install was done.
Then I have an ssh img that overlays that and places the correct keys for
the host being booted over those. The overlay ssh image also adds several
additional files in /etc/ssh.

If I get the OS container first, then get the ssh overlay, then the files
that are new from the overlay appear correctly, but files that should
overwrite those in the container don't appear so the new keys aren't there.

If I get the overlay img first, then the container, the files from the
overlay all get written correctly and the ones in the container do not
overwrite them.

Basically it behaves as if the first extracted img to write a file wins for
that file and all subsequent files with the same name are discarded. This
is counter-intuitive as I would have expected the last writer to win if I
had simply ran `cpio` to extract all the images to a single location.

Other than that bit of confusion, I haven't yet found an example of
anything missing that should be there in the resulting booted host
filesystem.

Despite it being confusing with first writer winning, it's still quite
useful behavior as collapsing all these overlay images before passing them
to the kernel simplifies the resulting kernel command line nicely.  Is it
worth adding a feature request for this if it's really unintentional? It's
trivial to use different names to keep them separate so if the merging
behavior was documented and stable, it'd be a nice option to have
available, especially if it could behave predictably like extracting a
series of cpio archives to the same location.

griznog


On Fri, Mar 11, 2022 at 12:54 PM Christian Iwata Nilsson <nikize at gmail.com>
wrote:

> On Fri, 11 Mar 2022 at 17:42, John Hanks <griznog at gmail.com> wrote:
>
>> Hi,
>>
>> I am using imgextract to download several images, like so
>>
>> imgextract --name one http://master/one.img
>> imgextract --name two http://master/two.img
>> imgextract --name three http://master/three.img
>> boot kernel initrd=initrd.magic initrd=one initrd=two initrd=three blah
>> blah blah
>>
>> So far so good, except that I wanted to try adding a lot of these but am
>> restricted by the maximum kernel command line length. I naively tried
>>
>> imgextract --name mysystem http://master/one.img
>> imgextract --name mysystem http://master/two.img
>> imgextract --name mysystem http://master/three.img
>> boot kernel initrd=initrd.magic initrd=mysystem blah blah blah
>>
>> and it worked!!! But, that seems way too easy, like I'm cheating. Is this
>> expected to work this way or am I exploiting a bug that might get fixed
>> later? I don't want to depend on this unless it's working this way on
>> purpose.
>>
>> Thanks,
>>
>> griznog
>>
>>
> Are you sure that the contents from them are all available?
> Since you are using imgextract I assume that the contents of *.img is
> compressed and that it contains CPIO archives?
>
> Could you verify with imgstat that size adds up if you do them to separate
> names and then try those 2 to the same file?
> I see nothing that indicates this as being expected behaviour.
>
> /Christian
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ipxe.org/pipermail/ipxe-devel/attachments/20220311/9cc24f37/attachment.htm>