[ipxe-devel] iPXE, ESXi 5.5 Stateless + Caching Install - BMP Razor + Chef Integration, Routed iSCSI, IaaS block

[Andrew Bobulsky]

Hello Mike,

On Tue, Jul 22, 2014 at 9:44 AM, Mike Harris <mharris at quovadis.bm> wrote:
> Greetings!
>
> I'm currently using local storage (2-way mirror, LSI controller) to booting
> a "test rabbit" SuperMicro blade in my lab.
>
> High level;
>
> + X8DTT-H Motherboard
> + Intel X540 NIC (dual 10G copper)
>
> We commonly use these blades and would like to bare metal provision a failed
> blade from a last known good state.  Final profile applied by an ESM tool
> like Chef.
>
> Currently the blade use DAS.  It works, but an iSRB is better and more
> convenient (from a hardware point of view).
>
> I would like to;
>
> + iPXE boot,
> + Attach an SAN volume for a boot device (1),
> + have Razor; factor, tag, kickstart the ESXi 5.5 install process.
> + Then broker the node to Chef for final provisioning.

Everything you've laid out here sounds pretty normal.  Though I can't
claim to know what "iSRB" is, and Google wasn't much help either ;)

> I've been able to get most of this working aside from the SAN volume (1).
> Routed iSCSI/NFS SAN volume is a challenge since the default iPXE binary
> doesn't support vcreate.

Are you working with a burned-in iPXE ROM, then?  Even with iPXE
built-in to your systems, you can still use that ROM to chainload a
more feature-complete iPXE binary from one of your servers.  Have a
look at the links here for an idea of how to do it with dhcpd:
http://ipxe.org/examples#using_chainloading_to_improve_the_feature_set_of_ipxe

Alternatively, you might even be able to do that with an iPXE script
that chainloads a different iPXE if the vcreate command fails.  Keep
in mind that this is just a guess, but I'd certainly try it if
adjusting my DHCP server was off the table.  Something like:

vcreate --tag 100 && goto attachMySAN || goto ipxeUpgrade

Some cursory testing shows me that it'd work: http://i.imgur.com/S0yPnix.png

But in the interest of Doing it Right[TM], something like this might
be most appropriate: http://pastebin.com/raw.php?i=bDV5M1X6

> I haven't found any exampled of routed iSCSI (or
> NFS), I'm sure someone has, hopefully they're on this mailing list.

Routed iSCSI/NFS?  Unless you mean something different... iPXE's iSCSI
and NFS clients shouldn't have a problem talking to a server in a
different subnet, assuming that there's no firewall/NAT/ALG/whatever
in the way to mess with your traffic.

> If anyone has any tips on routed iSCSI/NFS boot volumes, and Razor/Chef
> integration experience, I'd be move appreciative for some feedback on how
> you managed iSRB.  I have a pretty network diagram of the POC which I'm
> happy to share if you're interested.

Diagrams couldn't hurt, though if you're looking to limit the scope of
who sees them, chances are pretty high that exposing such things to my
eyeballs won't yield a significant return.  Even in spite of my
enthusiasm for iPXE sorcery ;)

> Although the reward is strictly karma at this point, I may have a bunch of
> Chef work that needs doing that could lead to some meaningful PS for a
> couple of ninjas in a cool location or two.
>
> May the force be with you!
>
> Mike

The force in strong in this list! ;)

Best Regards,
Andrew Bobulsky