|  | ATA over Ethernet is a network protocol that provides simple access to | 
|  | block storage on the LAN. | 
|  |  | 
|  | http://support.coraid.com/documents/AoEr11.txt | 
|  |  | 
|  | The EtherDrive (R) HOWTO for 2.6 and 3.x kernels is found at ... | 
|  |  | 
|  | http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO.html | 
|  |  | 
|  | It has many tips and hints!  Please see, especially, recommended | 
|  | tunings for virtual memory: | 
|  |  | 
|  | http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO-5.html#ss5.19 | 
|  |  | 
|  | The aoetools are userland programs that are designed to work with this | 
|  | driver.  The aoetools are on sourceforge. | 
|  |  | 
|  | http://aoetools.sourceforge.net/ | 
|  |  | 
|  | The scripts in this Documentation/aoe directory are intended to | 
|  | document the use of the driver and are not necessary if you install | 
|  | the aoetools. | 
|  |  | 
|  |  | 
|  | CREATING DEVICE NODES | 
|  |  | 
|  | Users of udev should find the block device nodes created | 
|  | automatically, but to create all the necessary device nodes, use the | 
|  | udev configuration rules provided in udev.txt (in this directory). | 
|  |  | 
|  | There is a udev-install.sh script that shows how to install these | 
|  | rules on your system. | 
|  |  | 
|  | There is also an autoload script that shows how to edit | 
|  | /etc/modprobe.d/aoe.conf to ensure that the aoe module is loaded when | 
|  | necessary.  Preloading the aoe module is preferable to autoloading, | 
|  | however, because AoE discovery takes a few seconds.  It can be | 
|  | confusing when an AoE device is not present the first time the a | 
|  | command is run but appears a second later. | 
|  |  | 
|  | USING DEVICE NODES | 
|  |  | 
|  | "cat /dev/etherd/err" blocks, waiting for error diagnostic output, | 
|  | like any retransmitted packets. | 
|  |  | 
|  | "echo eth2 eth4 > /dev/etherd/interfaces" tells the aoe driver to | 
|  | limit ATA over Ethernet traffic to eth2 and eth4.  AoE traffic from | 
|  | untrusted networks should be ignored as a matter of security.  See | 
|  | also the aoe_iflist driver option described below. | 
|  |  | 
|  | "echo > /dev/etherd/discover" tells the driver to find out what AoE | 
|  | devices are available. | 
|  |  | 
|  | In the future these character devices may disappear and be replaced | 
|  | by sysfs counterparts.  Using the commands in aoetools insulates | 
|  | users from these implementation details. | 
|  |  | 
|  | The block devices are named like this: | 
|  |  | 
|  | e{shelf}.{slot} | 
|  | e{shelf}.{slot}p{part} | 
|  |  | 
|  | ... so that "e0.2" is the third blade from the left (slot 2) in the | 
|  | first shelf (shelf address zero).  That's the whole disk.  The first | 
|  | partition on that disk would be "e0.2p1". | 
|  |  | 
|  | USING SYSFS | 
|  |  | 
|  | Each aoe block device in /sys/block has the extra attributes of | 
|  | state, mac, and netif.  The state attribute is "up" when the device | 
|  | is ready for I/O and "down" if detected but unusable.  The | 
|  | "down,closewait" state shows that the device is still open and | 
|  | cannot come up again until it has been closed. | 
|  |  | 
|  | The mac attribute is the ethernet address of the remote AoE device. | 
|  | The netif attribute is the network interface on the localhost | 
|  | through which we are communicating with the remote AoE device. | 
|  |  | 
|  | There is a script in this directory that formats this information in | 
|  | a convenient way.  Users with aoetools should use the aoe-stat | 
|  | command. | 
|  |  | 
|  | root@makki root# sh Documentation/aoe/status.sh | 
|  | e10.0            eth3              up | 
|  | e10.1            eth3              up | 
|  | e10.2            eth3              up | 
|  | e10.3            eth3              up | 
|  | e10.4            eth3              up | 
|  | e10.5            eth3              up | 
|  | e10.6            eth3              up | 
|  | e10.7            eth3              up | 
|  | e10.8            eth3              up | 
|  | e10.9            eth3              up | 
|  | e4.0            eth1              up | 
|  | e4.1            eth1              up | 
|  | e4.2            eth1              up | 
|  | e4.3            eth1              up | 
|  | e4.4            eth1              up | 
|  | e4.5            eth1              up | 
|  | e4.6            eth1              up | 
|  | e4.7            eth1              up | 
|  | e4.8            eth1              up | 
|  | e4.9            eth1              up | 
|  |  | 
|  | Use /sys/module/aoe/parameters/aoe_iflist (or better, the driver | 
|  | option discussed below) instead of /dev/etherd/interfaces to limit | 
|  | AoE traffic to the network interfaces in the given | 
|  | whitespace-separated list.  Unlike the old character device, the | 
|  | sysfs entry can be read from as well as written to. | 
|  |  | 
|  | It's helpful to trigger discovery after setting the list of allowed | 
|  | interfaces.  The aoetools package provides an aoe-discover script | 
|  | for this purpose.  You can also directly use the | 
|  | /dev/etherd/discover special file described above. | 
|  |  | 
|  | DRIVER OPTIONS | 
|  |  | 
|  | There is a boot option for the built-in aoe driver and a | 
|  | corresponding module parameter, aoe_iflist.  Without this option, | 
|  | all network interfaces may be used for ATA over Ethernet.  Here is a | 
|  | usage example for the module parameter. | 
|  |  | 
|  | modprobe aoe_iflist="eth1 eth3" | 
|  |  | 
|  | The aoe_deadsecs module parameter determines the maximum number of | 
|  | seconds that the driver will wait for an AoE device to provide a | 
|  | response to an AoE command.  After aoe_deadsecs seconds have | 
|  | elapsed, the AoE device will be marked as "down".  A value of zero | 
|  | is supported for testing purposes and makes the aoe driver keep | 
|  | trying AoE commands forever. | 
|  |  | 
|  | The aoe_maxout module parameter has a default of 128.  This is the | 
|  | maximum number of unresponded packets that will be sent to an AoE | 
|  | target at one time. | 
|  |  | 
|  | The aoe_dyndevs module parameter defaults to 1, meaning that the | 
|  | driver will assign a block device minor number to a discovered AoE | 
|  | target based on the order of its discovery.  With dynamic minor | 
|  | device numbers in use, a greater range of AoE shelf and slot | 
|  | addresses can be supported.  Users with udev will never have to | 
|  | think about minor numbers.  Using aoe_dyndevs=0 allows device nodes | 
|  | to be pre-created using a static minor-number scheme with the | 
|  | aoe-mkshelf script in the aoetools. |