Commit | Line | Data |
---|---|---|
e3e0a28b HK |
1 | <?xml version="1.0" encoding="UTF-8"?> |
2 | <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" | |
3 | "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" []> | |
4 | ||
5 | <book id="index"> | |
6 | <bookinfo> | |
7 | <title>The Userspace I/O HOWTO</title> | |
8 | ||
9 | <author> | |
10 | <firstname>Hans-Jürgen</firstname> | |
11 | <surname>Koch</surname> | |
12 | <authorblurb><para>Linux developer, Linutronix</para></authorblurb> | |
13 | <affiliation> | |
14 | <orgname> | |
15 | <ulink url="http://www.linutronix.de">Linutronix</ulink> | |
16 | </orgname> | |
17 | ||
18 | <address> | |
19 | <email>hjk@linutronix.de</email> | |
20 | </address> | |
21 | </affiliation> | |
22 | </author> | |
23 | ||
24 | <pubdate>2006-12-11</pubdate> | |
25 | ||
26 | <abstract> | |
27 | <para>This HOWTO describes concept and usage of Linux kernel's | |
28 | Userspace I/O system.</para> | |
29 | </abstract> | |
30 | ||
31 | <revhistory> | |
ed423c24 HK |
32 | <revision> |
33 | <revnumber>0.4</revnumber> | |
34 | <date>2007-11-26</date> | |
35 | <authorinitials>hjk</authorinitials> | |
36 | <revremark>Removed section about uio_dummy.</revremark> | |
37 | </revision> | |
e3e0a28b HK |
38 | <revision> |
39 | <revnumber>0.3</revnumber> | |
40 | <date>2007-04-29</date> | |
41 | <authorinitials>hjk</authorinitials> | |
42 | <revremark>Added section about userspace drivers.</revremark> | |
43 | </revision> | |
44 | <revision> | |
45 | <revnumber>0.2</revnumber> | |
46 | <date>2007-02-13</date> | |
47 | <authorinitials>hjk</authorinitials> | |
48 | <revremark>Update after multiple mappings were added.</revremark> | |
49 | </revision> | |
50 | <revision> | |
51 | <revnumber>0.1</revnumber> | |
52 | <date>2006-12-11</date> | |
53 | <authorinitials>hjk</authorinitials> | |
54 | <revremark>First draft.</revremark> | |
55 | </revision> | |
56 | </revhistory> | |
57 | </bookinfo> | |
58 | ||
59 | <chapter id="aboutthisdoc"> | |
60 | <?dbhtml filename="about.html"?> | |
61 | <title>About this document</title> | |
62 | ||
63 | <sect1 id="copyright"> | |
64 | <?dbhtml filename="copyright.html"?> | |
65 | <title>Copyright and License</title> | |
66 | <para> | |
67 | Copyright (c) 2006 by Hans-Jürgen Koch.</para> | |
68 | <para> | |
69 | This documentation is Free Software licensed under the terms of the | |
70 | GPL version 2. | |
71 | </para> | |
72 | </sect1> | |
73 | ||
74 | <sect1 id="translations"> | |
75 | <?dbhtml filename="translations.html"?> | |
76 | <title>Translations</title> | |
77 | ||
78 | <para>If you know of any translations for this document, or you are | |
79 | interested in translating it, please email me | |
80 | <email>hjk@linutronix.de</email>. | |
81 | </para> | |
82 | </sect1> | |
83 | ||
84 | <sect1 id="preface"> | |
85 | <title>Preface</title> | |
86 | <para> | |
87 | For many types of devices, creating a Linux kernel driver is | |
88 | overkill. All that is really needed is some way to handle an | |
89 | interrupt and provide access to the memory space of the | |
90 | device. The logic of controlling the device does not | |
91 | necessarily have to be within the kernel, as the device does | |
92 | not need to take advantage of any of other resources that the | |
93 | kernel provides. One such common class of devices that are | |
94 | like this are for industrial I/O cards. | |
95 | </para> | |
96 | <para> | |
97 | To address this situation, the userspace I/O system (UIO) was | |
98 | designed. For typical industrial I/O cards, only a very small | |
99 | kernel module is needed. The main part of the driver will run in | |
100 | user space. This simplifies development and reduces the risk of | |
101 | serious bugs within a kernel module. | |
102 | </para> | |
ed423c24 HK |
103 | <para> |
104 | Please note that UIO is not an universal driver interface. Devices | |
105 | that are already handled well by other kernel subsystems (like | |
106 | networking or serial or USB) are no candidates for an UIO driver. | |
107 | Hardware that is ideally suited for an UIO driver fulfills all of | |
108 | the following: | |
109 | </para> | |
110 | <itemizedlist> | |
111 | <listitem> | |
112 | <para>The device has memory that can be mapped. The device can be | |
113 | controlled completely by writing to this memory.</para> | |
114 | </listitem> | |
115 | <listitem> | |
116 | <para>The device usually generates interrupts.</para> | |
117 | </listitem> | |
118 | <listitem> | |
119 | <para>The device does not fit into one of the standard kernel | |
120 | subsystems.</para> | |
121 | </listitem> | |
122 | </itemizedlist> | |
e3e0a28b HK |
123 | </sect1> |
124 | ||
125 | <sect1 id="thanks"> | |
126 | <title>Acknowledgments</title> | |
127 | <para>I'd like to thank Thomas Gleixner and Benedikt Spranger of | |
128 | Linutronix, who have not only written most of the UIO code, but also | |
129 | helped greatly writing this HOWTO by giving me all kinds of background | |
130 | information.</para> | |
131 | </sect1> | |
132 | ||
133 | <sect1 id="feedback"> | |
134 | <title>Feedback</title> | |
135 | <para>Find something wrong with this document? (Or perhaps something | |
136 | right?) I would love to hear from you. Please email me at | |
137 | <email>hjk@linutronix.de</email>.</para> | |
138 | </sect1> | |
139 | </chapter> | |
140 | ||
141 | <chapter id="about"> | |
142 | <?dbhtml filename="about.html"?> | |
143 | <title>About UIO</title> | |
144 | ||
145 | <para>If you use UIO for your card's driver, here's what you get:</para> | |
146 | ||
147 | <itemizedlist> | |
148 | <listitem> | |
149 | <para>only one small kernel module to write and maintain.</para> | |
150 | </listitem> | |
151 | <listitem> | |
152 | <para>develop the main part of your driver in user space, | |
153 | with all the tools and libraries you're used to.</para> | |
154 | </listitem> | |
155 | <listitem> | |
156 | <para>bugs in your driver won't crash the kernel.</para> | |
157 | </listitem> | |
158 | <listitem> | |
159 | <para>updates of your driver can take place without recompiling | |
160 | the kernel.</para> | |
161 | </listitem> | |
e3e0a28b HK |
162 | </itemizedlist> |
163 | ||
164 | <sect1 id="how_uio_works"> | |
165 | <title>How UIO works</title> | |
166 | <para> | |
167 | Each UIO device is accessed through a device file and several | |
168 | sysfs attribute files. The device file will be called | |
169 | <filename>/dev/uio0</filename> for the first device, and | |
170 | <filename>/dev/uio1</filename>, <filename>/dev/uio2</filename> | |
171 | and so on for subsequent devices. | |
172 | </para> | |
173 | ||
174 | <para><filename>/dev/uioX</filename> is used to access the | |
175 | address space of the card. Just use | |
176 | <function>mmap()</function> to access registers or RAM | |
177 | locations of your card. | |
178 | </para> | |
179 | ||
180 | <para> | |
181 | Interrupts are handled by reading from | |
182 | <filename>/dev/uioX</filename>. A blocking | |
183 | <function>read()</function> from | |
184 | <filename>/dev/uioX</filename> will return as soon as an | |
185 | interrupt occurs. You can also use | |
186 | <function>select()</function> on | |
187 | <filename>/dev/uioX</filename> to wait for an interrupt. The | |
188 | integer value read from <filename>/dev/uioX</filename> | |
189 | represents the total interrupt count. You can use this number | |
190 | to figure out if you missed some interrupts. | |
191 | </para> | |
192 | ||
193 | <para> | |
194 | To handle interrupts properly, your custom kernel module can | |
195 | provide its own interrupt handler. It will automatically be | |
196 | called by the built-in handler. | |
197 | </para> | |
198 | ||
199 | <para> | |
200 | For cards that don't generate interrupts but need to be | |
201 | polled, there is the possibility to set up a timer that | |
202 | triggers the interrupt handler at configurable time intervals. | |
ed423c24 HK |
203 | This interrupt simulation is done by calling |
204 | <function>uio_event_notify()</function> | |
205 | from the timer's event handler. | |
e3e0a28b HK |
206 | </para> |
207 | ||
208 | <para> | |
209 | Each driver provides attributes that are used to read or write | |
210 | variables. These attributes are accessible through sysfs | |
211 | files. A custom kernel driver module can add its own | |
212 | attributes to the device owned by the uio driver, but not added | |
213 | to the UIO device itself at this time. This might change in the | |
214 | future if it would be found to be useful. | |
215 | </para> | |
216 | ||
217 | <para> | |
218 | The following standard attributes are provided by the UIO | |
219 | framework: | |
220 | </para> | |
221 | <itemizedlist> | |
222 | <listitem> | |
223 | <para> | |
224 | <filename>name</filename>: The name of your device. It is | |
225 | recommended to use the name of your kernel module for this. | |
226 | </para> | |
227 | </listitem> | |
228 | <listitem> | |
229 | <para> | |
230 | <filename>version</filename>: A version string defined by your | |
231 | driver. This allows the user space part of your driver to deal | |
232 | with different versions of the kernel module. | |
233 | </para> | |
234 | </listitem> | |
235 | <listitem> | |
236 | <para> | |
237 | <filename>event</filename>: The total number of interrupts | |
238 | handled by the driver since the last time the device node was | |
239 | read. | |
240 | </para> | |
241 | </listitem> | |
242 | </itemizedlist> | |
243 | <para> | |
244 | These attributes appear under the | |
245 | <filename>/sys/class/uio/uioX</filename> directory. Please | |
246 | note that this directory might be a symlink, and not a real | |
247 | directory. Any userspace code that accesses it must be able | |
248 | to handle this. | |
249 | </para> | |
250 | <para> | |
251 | Each UIO device can make one or more memory regions available for | |
252 | memory mapping. This is necessary because some industrial I/O cards | |
253 | require access to more than one PCI memory region in a driver. | |
254 | </para> | |
255 | <para> | |
256 | Each mapping has its own directory in sysfs, the first mapping | |
257 | appears as <filename>/sys/class/uio/uioX/maps/map0/</filename>. | |
258 | Subsequent mappings create directories <filename>map1/</filename>, | |
259 | <filename>map2/</filename>, and so on. These directories will only | |
260 | appear if the size of the mapping is not 0. | |
261 | </para> | |
262 | <para> | |
263 | Each <filename>mapX/</filename> directory contains two read-only files | |
264 | that show start address and size of the memory: | |
265 | </para> | |
266 | <itemizedlist> | |
267 | <listitem> | |
268 | <para> | |
269 | <filename>addr</filename>: The address of memory that can be mapped. | |
270 | </para> | |
271 | </listitem> | |
272 | <listitem> | |
273 | <para> | |
274 | <filename>size</filename>: The size, in bytes, of the memory | |
275 | pointed to by addr. | |
276 | </para> | |
277 | </listitem> | |
278 | </itemizedlist> | |
279 | ||
280 | <para> | |
281 | From userspace, the different mappings are distinguished by adjusting | |
282 | the <varname>offset</varname> parameter of the | |
283 | <function>mmap()</function> call. To map the memory of mapping N, you | |
284 | have to use N times the page size as your offset: | |
285 | </para> | |
286 | <programlisting format="linespecific"> | |
287 | offset = N * getpagesize(); | |
288 | </programlisting> | |
289 | ||
290 | </sect1> | |
291 | </chapter> | |
292 | ||
e3e0a28b HK |
293 | <chapter id="custom_kernel_module" xreflabel="Writing your own kernel module"> |
294 | <?dbhtml filename="custom_kernel_module.html"?> | |
295 | <title>Writing your own kernel module</title> | |
296 | <para> | |
ed423c24 | 297 | Please have a look at <filename>uio_cif.c</filename> as an |
e3e0a28b HK |
298 | example. The following paragraphs explain the different |
299 | sections of this file. | |
300 | </para> | |
301 | ||
302 | <sect1 id="uio_info"> | |
303 | <title>struct uio_info</title> | |
304 | <para> | |
305 | This structure tells the framework the details of your driver, | |
306 | Some of the members are required, others are optional. | |
307 | </para> | |
308 | ||
309 | <itemizedlist> | |
310 | <listitem><para> | |
311 | <varname>char *name</varname>: Required. The name of your driver as | |
312 | it will appear in sysfs. I recommend using the name of your module for this. | |
313 | </para></listitem> | |
314 | ||
315 | <listitem><para> | |
316 | <varname>char *version</varname>: Required. This string appears in | |
317 | <filename>/sys/class/uio/uioX/version</filename>. | |
318 | </para></listitem> | |
319 | ||
320 | <listitem><para> | |
321 | <varname>struct uio_mem mem[ MAX_UIO_MAPS ]</varname>: Required if you | |
322 | have memory that can be mapped with <function>mmap()</function>. For each | |
323 | mapping you need to fill one of the <varname>uio_mem</varname> structures. | |
324 | See the description below for details. | |
325 | </para></listitem> | |
326 | ||
327 | <listitem><para> | |
328 | <varname>long irq</varname>: Required. If your hardware generates an | |
329 | interrupt, it's your modules task to determine the irq number during | |
330 | initialization. If you don't have a hardware generated interrupt but | |
331 | want to trigger the interrupt handler in some other way, set | |
ed423c24 HK |
332 | <varname>irq</varname> to <varname>UIO_IRQ_CUSTOM</varname>. |
333 | If you had no interrupt at all, you could set | |
e3e0a28b HK |
334 | <varname>irq</varname> to <varname>UIO_IRQ_NONE</varname>, though this |
335 | rarely makes sense. | |
336 | </para></listitem> | |
337 | ||
338 | <listitem><para> | |
339 | <varname>unsigned long irq_flags</varname>: Required if you've set | |
340 | <varname>irq</varname> to a hardware interrupt number. The flags given | |
341 | here will be used in the call to <function>request_irq()</function>. | |
342 | </para></listitem> | |
343 | ||
344 | <listitem><para> | |
345 | <varname>int (*mmap)(struct uio_info *info, struct vm_area_struct | |
346 | *vma)</varname>: Optional. If you need a special | |
347 | <function>mmap()</function> function, you can set it here. If this | |
348 | pointer is not NULL, your <function>mmap()</function> will be called | |
349 | instead of the built-in one. | |
350 | </para></listitem> | |
351 | ||
352 | <listitem><para> | |
353 | <varname>int (*open)(struct uio_info *info, struct inode *inode) | |
354 | </varname>: Optional. You might want to have your own | |
355 | <function>open()</function>, e.g. to enable interrupts only when your | |
356 | device is actually used. | |
357 | </para></listitem> | |
358 | ||
359 | <listitem><para> | |
360 | <varname>int (*release)(struct uio_info *info, struct inode *inode) | |
361 | </varname>: Optional. If you define your own | |
362 | <function>open()</function>, you will probably also want a custom | |
363 | <function>release()</function> function. | |
364 | </para></listitem> | |
365 | </itemizedlist> | |
366 | ||
367 | <para> | |
368 | Usually, your device will have one or more memory regions that can be mapped | |
369 | to user space. For each region, you have to set up a | |
370 | <varname>struct uio_mem</varname> in the <varname>mem[]</varname> array. | |
371 | Here's a description of the fields of <varname>struct uio_mem</varname>: | |
372 | </para> | |
373 | ||
374 | <itemizedlist> | |
375 | <listitem><para> | |
376 | <varname>int memtype</varname>: Required if the mapping is used. Set this to | |
377 | <varname>UIO_MEM_PHYS</varname> if you you have physical memory on your | |
378 | card to be mapped. Use <varname>UIO_MEM_LOGICAL</varname> for logical | |
379 | memory (e.g. allocated with <function>kmalloc()</function>). There's also | |
380 | <varname>UIO_MEM_VIRTUAL</varname> for virtual memory. | |
381 | </para></listitem> | |
382 | ||
383 | <listitem><para> | |
384 | <varname>unsigned long addr</varname>: Required if the mapping is used. | |
385 | Fill in the address of your memory block. This address is the one that | |
386 | appears in sysfs. | |
387 | </para></listitem> | |
388 | ||
389 | <listitem><para> | |
390 | <varname>unsigned long size</varname>: Fill in the size of the | |
391 | memory block that <varname>addr</varname> points to. If <varname>size</varname> | |
392 | is zero, the mapping is considered unused. Note that you | |
393 | <emphasis>must</emphasis> initialize <varname>size</varname> with zero for | |
394 | all unused mappings. | |
395 | </para></listitem> | |
396 | ||
397 | <listitem><para> | |
398 | <varname>void *internal_addr</varname>: If you have to access this memory | |
399 | region from within your kernel module, you will want to map it internally by | |
400 | using something like <function>ioremap()</function>. Addresses | |
401 | returned by this function cannot be mapped to user space, so you must not | |
402 | store it in <varname>addr</varname>. Use <varname>internal_addr</varname> | |
403 | instead to remember such an address. | |
404 | </para></listitem> | |
405 | </itemizedlist> | |
406 | ||
407 | <para> | |
408 | Please do not touch the <varname>kobj</varname> element of | |
409 | <varname>struct uio_mem</varname>! It is used by the UIO framework | |
410 | to set up sysfs files for this mapping. Simply leave it alone. | |
411 | </para> | |
412 | </sect1> | |
413 | ||
414 | <sect1 id="adding_irq_handler"> | |
415 | <title>Adding an interrupt handler</title> | |
416 | <para> | |
417 | What you need to do in your interrupt handler depends on your | |
418 | hardware and on how you want to handle it. You should try to | |
419 | keep the amount of code in your kernel interrupt handler low. | |
420 | If your hardware requires no action that you | |
421 | <emphasis>have</emphasis> to perform after each interrupt, | |
422 | then your handler can be empty.</para> <para>If, on the other | |
423 | hand, your hardware <emphasis>needs</emphasis> some action to | |
424 | be performed after each interrupt, then you | |
425 | <emphasis>must</emphasis> do it in your kernel module. Note | |
426 | that you cannot rely on the userspace part of your driver. Your | |
427 | userspace program can terminate at any time, possibly leaving | |
428 | your hardware in a state where proper interrupt handling is | |
429 | still required. | |
430 | </para> | |
431 | ||
432 | <para> | |
433 | There might also be applications where you want to read data | |
434 | from your hardware at each interrupt and buffer it in a piece | |
435 | of kernel memory you've allocated for that purpose. With this | |
436 | technique you could avoid loss of data if your userspace | |
437 | program misses an interrupt. | |
438 | </para> | |
439 | ||
440 | <para> | |
441 | A note on shared interrupts: Your driver should support | |
442 | interrupt sharing whenever this is possible. It is possible if | |
443 | and only if your driver can detect whether your hardware has | |
444 | triggered the interrupt or not. This is usually done by looking | |
445 | at an interrupt status register. If your driver sees that the | |
446 | IRQ bit is actually set, it will perform its actions, and the | |
447 | handler returns IRQ_HANDLED. If the driver detects that it was | |
448 | not your hardware that caused the interrupt, it will do nothing | |
449 | and return IRQ_NONE, allowing the kernel to call the next | |
450 | possible interrupt handler. | |
451 | </para> | |
452 | ||
453 | <para> | |
454 | If you decide not to support shared interrupts, your card | |
455 | won't work in computers with no free interrupts. As this | |
456 | frequently happens on the PC platform, you can save yourself a | |
457 | lot of trouble by supporting interrupt sharing. | |
458 | </para> | |
459 | </sect1> | |
460 | ||
461 | </chapter> | |
462 | ||
463 | <chapter id="userspace_driver" xreflabel="Writing a driver in user space"> | |
464 | <?dbhtml filename="userspace_driver.html"?> | |
465 | <title>Writing a driver in userspace</title> | |
466 | <para> | |
467 | Once you have a working kernel module for your hardware, you can | |
468 | write the userspace part of your driver. You don't need any special | |
469 | libraries, your driver can be written in any reasonable language, | |
470 | you can use floating point numbers and so on. In short, you can | |
471 | use all the tools and libraries you'd normally use for writing a | |
472 | userspace application. | |
473 | </para> | |
474 | ||
475 | <sect1 id="getting_uio_information"> | |
476 | <title>Getting information about your UIO device</title> | |
477 | <para> | |
478 | Information about all UIO devices is available in sysfs. The | |
479 | first thing you should do in your driver is check | |
480 | <varname>name</varname> and <varname>version</varname> to | |
481 | make sure your talking to the right device and that its kernel | |
482 | driver has the version you expect. | |
483 | </para> | |
484 | <para> | |
485 | You should also make sure that the memory mapping you need | |
486 | exists and has the size you expect. | |
487 | </para> | |
488 | <para> | |
489 | There is a tool called <varname>lsuio</varname> that lists | |
490 | UIO devices and their attributes. It is available here: | |
491 | </para> | |
492 | <para> | |
493 | <ulink url="http://www.osadl.org/projects/downloads/UIO/user/"> | |
494 | http://www.osadl.org/projects/downloads/UIO/user/</ulink> | |
495 | </para> | |
496 | <para> | |
497 | With <varname>lsuio</varname> you can quickly check if your | |
498 | kernel module is loaded and which attributes it exports. | |
499 | Have a look at the manpage for details. | |
500 | </para> | |
501 | <para> | |
502 | The source code of <varname>lsuio</varname> can serve as an | |
503 | example for getting information about an UIO device. | |
504 | The file <filename>uio_helper.c</filename> contains a lot of | |
505 | functions you could use in your userspace driver code. | |
506 | </para> | |
507 | </sect1> | |
508 | ||
509 | <sect1 id="mmap_device_memory"> | |
510 | <title>mmap() device memory</title> | |
511 | <para> | |
512 | After you made sure you've got the right device with the | |
513 | memory mappings you need, all you have to do is to call | |
514 | <function>mmap()</function> to map the device's memory | |
515 | to userspace. | |
516 | </para> | |
517 | <para> | |
518 | The parameter <varname>offset</varname> of the | |
519 | <function>mmap()</function> call has a special meaning | |
520 | for UIO devices: It is used to select which mapping of | |
521 | your device you want to map. To map the memory of | |
522 | mapping N, you have to use N times the page size as | |
523 | your offset: | |
524 | </para> | |
525 | <programlisting format="linespecific"> | |
526 | offset = N * getpagesize(); | |
527 | </programlisting> | |
528 | <para> | |
529 | N starts from zero, so if you've got only one memory | |
530 | range to map, set <varname>offset = 0</varname>. | |
531 | A drawback of this technique is that memory is always | |
532 | mapped beginning with its start address. | |
533 | </para> | |
534 | </sect1> | |
535 | ||
536 | <sect1 id="wait_for_interrupts"> | |
537 | <title>Waiting for interrupts</title> | |
538 | <para> | |
539 | After you successfully mapped your devices memory, you | |
540 | can access it like an ordinary array. Usually, you will | |
541 | perform some initialization. After that, your hardware | |
542 | starts working and will generate an interrupt as soon | |
543 | as it's finished, has some data available, or needs your | |
544 | attention because an error occured. | |
545 | </para> | |
546 | <para> | |
547 | <filename>/dev/uioX</filename> is a read-only file. A | |
548 | <function>read()</function> will always block until an | |
549 | interrupt occurs. There is only one legal value for the | |
550 | <varname>count</varname> parameter of | |
551 | <function>read()</function>, and that is the size of a | |
552 | signed 32 bit integer (4). Any other value for | |
553 | <varname>count</varname> causes <function>read()</function> | |
554 | to fail. The signed 32 bit integer read is the interrupt | |
555 | count of your device. If the value is one more than the value | |
556 | you read the last time, everything is OK. If the difference | |
557 | is greater than one, you missed interrupts. | |
558 | </para> | |
559 | <para> | |
560 | You can also use <function>select()</function> on | |
561 | <filename>/dev/uioX</filename>. | |
562 | </para> | |
563 | </sect1> | |
564 | ||
565 | </chapter> | |
566 | ||
567 | <appendix id="app1"> | |
568 | <title>Further information</title> | |
569 | <itemizedlist> | |
570 | <listitem><para> | |
571 | <ulink url="http://www.osadl.org"> | |
572 | OSADL homepage.</ulink> | |
573 | </para></listitem> | |
574 | <listitem><para> | |
575 | <ulink url="http://www.linutronix.de"> | |
576 | Linutronix homepage.</ulink> | |
577 | </para></listitem> | |
578 | </itemizedlist> | |
579 | </appendix> | |
580 | ||
581 | </book> |