[ipxe-devel] Native Ceph Connectivity

[Michael Brown]

On 13/05/14 16:23, Stephen Perkins wrote:
> But… the hard part is that I want to boot diskless XenServers from this
> ceph store.   This is where:
>
>                  1) iPXE comes in
>
>                  2) My knowledge about ends
>
> My thought is that I would boot a customized iPXE from a tiny USB DOM or
> a highly available DHCP/TFTP/PXEboot infrastructure.  Once iPXE is
> running, I would like to boot directly from a ceph cluster volume
> instead of the more standard iSCSI volume.
>
> So, I wanted to ask thoughts on whether it makes sense to try to develop
> another backend connectivity option to iPXE.  I would like to look at
> adding a ceph/RBD option that will allow me to provide a list of IP
> addresses (and other needed config info) for the ceph cluster and allow
> me to mount a ceph store to boot from.

Without knowing any details of the ceph protocol, it seems reasonable to 
me.  iPXE already supports multiple block storage backends (iSCSI, AoE, 
SRP, HTTP), and the architecture allows for other protocols to be added 
cleanly.

Apart from basic TCP/IP connectivity, does ceph require anything special 
at the protocol level?  Possible hurdles I can think of are:

  - Authentication - does ceph allow anonymous access?

  - Multicast - as a distributed system, do we need to be able to join 
multicast groups?  We have no support for that at present.

> Then… I would have to address the problem of how to get an initrd for
> XenServer that would have the ceph modules available and allow me to
> boot a root file system from there.

You also need to consider a handover mechanism: how does the bootloader 
communicate information about the ceph store to the loaded OS?  (For 
iSCSI, this is done via the iBFT.)

> Is this a hugely monumental project… or just a monumental project (given
> that the ceph client code exists and is open source)…

Depends entirely on how complex the protocol is on the wire.  Your 
chances of reusing code from the ceph client are approximately zero 
given the different programming environment and size constraints, so it 
would be a case of creating a clean implementation based on a spec 
and/or traffic samples.

Michael