[ipxe-devel] reproducible builds

[Geert Stappers]

On Fri, May 01, 2020 at 06:36:05PM -0400, Neil Roza wrote:
> Hi ipxe-devel,

Hi

(and welcome  Neil Roza)


> Please find the attached diff representing a patch I would like to submit
> for your consideration. This is a small change to the
> `src/Makefile.housekeeping` that makes the generation of most artifacts
> (notably not `*.usb` images) deterministic.
> 
> The scariest change here is the removal of the `BUILD_ID_CMD` in favor of
> an inlined shell snippet where the `_build_id` symbol is defined. In
> keeping with the comments that specify a unique `_build_id` for each
> `$(BIN)/%.tmp`, I use the first 8 characters of the md5sum of the target,
> in the expected base-prefixed hexadecimal representation. Calculating the
> likelihood of collisions I leave as an exercise to the reviewer. :D
> 
> The `BUILD_TIMESTAMP` assignment has been changed to allow environment
> variable overriding, but it defaults to `SOURCE_DATE_EPOCH`. The source
> date epoch can also be overridden; it defaults to the Unix timestamp of the
> current git HEAD commit.
> 
> I like reproducible builds, but I recognize that others have different
> concerns. I'm happy to change what needs changing.
> 

I also like reproducible builds.  I'm happy to help find consensus.

> 
> -- 
> Neil Roza


That I missed something is concern for later ...

> diff --git a/src/Makefile.housekeeping b/src/Makefile.housekeeping
> index 1dd14794..93c598d2 100644
> --- a/src/Makefile.housekeeping
> +++ b/src/Makefile.housekeeping
> @@ -1163,14 +1163,22 @@ $(BLIB) : $(BLIB_OBJS) $(BLIB_LIST) $(MAKEDEPS)
>  	$(Q)$(RANLIB) $@
>  blib : $(BLIB)
>  
> -# Command to generate build ID.  Must be unique for each $(BIN)/%.tmp,
> -# even within the same build run.
> +# Source date epoch
>  #
> -BUILD_ID_CMD	:= perl -e 'printf "0x%08x", int ( rand ( 0xffffffff ) );'
> +# Assumptions:
> +# * the first element in MAKEFILE_LIST is src/Makefile
> +# * we want the unix timestamp for the commit on the current git HEAD
> +#
> +# References:
> +# * https://reproducible-builds.org/specs/source-date-epoch/
> +# * https://www.mankier.com/1/git-show
> +#
> +IPXE_DIR := $(abspath $(dir $(abspath $(firstword $(MAKEFILE_LIST))))/..)
> +SOURCE_DATE_EPOCH ?= $(shell git -C $(IPXE_DIR) show -s --format=%ct HEAD)
>  
>  # Build timestamp
>  #
> -BUILD_TIMESTAMP := $(shell date +%s)
> +BUILD_TIMESTAMP ?= $(SOURCE_DATE_EPOCH)
>  
>  # Build version
>  #
> @@ -1187,10 +1195,13 @@ $(BIN)/version.%.o : core/version.c $(MAKEDEPS) $(GIT_INDEX)
>  # Build an intermediate object file from the objects required for the
>  # specified target.
>  #
> +# Note: each _build_id must be unique for each $(BIN)/%.tmp, even within the
> +# same build run.
> +#
>  $(BIN)/%.tmp : $(BIN)/version.%.o $(BLIB) $(MAKEDEPS) $(LDSCRIPT)
>  	$(QM)$(ECHO) "  [LD] $@"
>  	$(Q)$(LD) $(LDFLAGS) -T $(LDSCRIPT) $(TGT_LD_FLAGS) $< $(BLIB) -o $@ \
> -		--defsym _build_id=`$(BUILD_ID_CMD)` \
> +		--defsym _build_id="0x$$(echo $@ | md5sum | head -c8)" \
>  		--defsym _build_timestamp=$(BUILD_TIMESTAMP) \
>  		-Map $(BIN)/$*.tmp.map
>  	$(Q)$(OBJDUMP) -ht $@ | $(PERL) $(SORTOBJDUMP) >> $(BIN)/$*.tmp.map


Oops, hefty changes.  I think I can make the proposed changes less intrusive.

To be contineued ...


Groeten
Geert Stappers
-- 
Silence is hard to parse