Yuma Developer Manual

YANG-Based Unified Modular Automation Tools

Server Instrumentation Library Development

Version 2.2

Last Updated: January 31, 2012

Table Of Contents

Yuma Developer Manual

1 Preface        4

1.1 Legal Statements        4

1.2 Additional Resources        4

1.2.1 WEB Sites        4

1.2.2 Mailing Lists        5

1.3 Conventions Used in this Document        5

2 Software Overview        6

2.1 Introduction        6

2.1.1 Intended Audience        6

2.1.2 What does Yuma Do?        6

2.1.3 What is a Yuma Root?        7

2.1.4 Searching Yuma Roots        8

2.1.5 What is a SIL?        9

2.1.6 Auto-generated SIL Files        9

2.1.7 Basic Development Steps        10

2.2 Yuma Source Files        11

2.2.1 src/ncx Directory        11

2.2.2 src/platform Directory        13

2.2.3 src/agt Directory        14

2.2.4 src/mgr Directory        15

2.2.5 src/subsys Directory        16

2.2.6 src/netconfd Directory        16

2.2.7 src/yangcli Directory        16

2.2.8 src/yangdiff Directory        16

2.2.9 src/yangdump Directory        17

2.3 Server Design        17

2.3.1 YANG Native Operation        18

2.3.2 YANG Object Tree        20

2.3.3 YANG Data Tree        21

2.3.4 Service Layering        22

2.3.5 Session Control Block        23

2.3.6 Server Message Flows        23

2.3.7 Main ncxserver Loop        25

2.3.8 SIL Callback Functions        26

2.4 Server Operation        27

2.4.1 Initialization        27

2.4.2 Loading Modules and SIL Code        28

2.4.3 Core Module Initialization        29

2.4.4 Startup Configuration Processing        29

2.4.5 Process an Incoming <rpc> Request        30

2.4.6 Edit the Database        31

2.4.7 Save the Database        33

2.5 Built-in Server Modules        33

2.5.1 ietf-inet-types.yang        33

2.5.2 ietf-netconf-monitoring.yang        34

2.5.3 ietf-with-defaults.yang        34

2.5.4 ietf-yang-types.yang        34

2.5.5 nc-notifications.yang        34

2.5.6 notifications.yang        34

2.5.7 yuma-app-common.yang        35

2.5.8 yuma-interfaces.yang        35

2.5.9 yuma-mysession.yang        35

2.5.10 yuma-nacm.yang        35

2.5.11 yuma-ncx.yang        35

2.5.12 yuma-netconf.yang        35

2.5.13 yuma-proc.yang        36

2.5.14 yuma-system.yang        36

2.5.15 yuma-time-filter.yang         36

2.5.16 yuma-types.yang        36

3 YANG Objects and Data Nodes        36

3.1 Object Definition Tree        36

3.1.1 Object Node Types        36

3.1.2 Object Node Template (obj_template_t)        38

3.1.3 obj_template_t Access Functions        39

3.2 Data Tree        42

3.2.1 Data Node Types        43

3.2.2 Yuma Data Node Edit Variables (val_editvars_t)        44

3.2.3 Yuma Data Nodes (val_value_t)        45

3.2.4 val_value_t Access Macros        49

3.2.5 val_value_t Access Functions        50

3.2.6 SIL Utility Functions        55

4 SIL External Interface        56

4.1 Stage 1 Initialization        56

4.2 Stage 2 Initialization        59

4.3 Cleanup        60

5 SIL Callback Interface        61

5.1 RPC Operation Interface        62

5.1.1 RPC Callback Initialization        62

5.1.2 RPC Message Header        63

5.1.3 SIL Support Functions For RPC Operations        66

5.1.4 RPC Validate Callback Function        66

5.1.5 RPC Invoke Callback Function        69

5.1.6 RPC Post Reply Callback Function        73

5.2 Database Operations        75

5.2.1 Database Template (cfg_template_t)        76

5.2.2 Database Access Functions        77

5.2.3 Database Callback Initialization and Cleanup        78

5.2.4 Example SIL Database Edit Callback Function        81

5.2.5 Database Edit Validate Callback Phase        83

5.2.6 Database Edit Apply Callback Phase        83

5.2.7 Database Edit Commit Callback Phase        83

5.2.8 Database Edit Rollback Callback Phase        84

5.2.9 Database Virtual Node Get Callback Function        84

5.3 Notifications        86

5.3.1 Notification Send Function        87

5.4 Periodic Timer Service        88

5.4.1 Timer Callback Function        88

5.4.2 Timer Access Functions        89

5.4.3 Example Timer Callback Function        89

6 Server Callback Examples        90

6.1 YANG        90

6.2 Edit Operations        91

6.2.1 Create an XPO container        91

6.2.2 Create a Profile        92

6.2.3 Create a Stream Connection        94

6.2.4 Delete an XPO Container        97

6.2.5 Delete a Profile        99

6.2.6 Delete a Stream Connection        101

7 Development Environment        101

7.1 Programs and Libraries Needed        101

7.2 SIL Makefile        102

7.2.1 Target Platforms        102

7.2.2 Build Targets        102

7.2.3 Command Line Build Options        103

7.2.4 Example SIL Makefile        104

7.3 Automation Control        108

7.3.1 Built-in YANG Language Extensions        109

7.3.2 SIL Language Extension Access Functions        110

1 Preface

1.1 Legal Statements

Copyright 2009 – 2012,  Andy Bierman,  All Rights Reserved.

1.2 Additional Resources

This document assumes you have successfully set up the software as described in the printed document:

        Yuma  Installation Guide

        Yuma Quickstart Guide

Other documentation includes:

        Yuma User Manual

        Yuma  netconfd  Manual

        Yuma  yangcli Manual

        Yuma  yangdiff Manual

To obtain additional support you may join the yuma-users group on sourceforge.net and send email to this e-mail address:

 yuma-users@lists.sourceforge.net

The SourceForge.net Support Page for Yuma can be found at this WEB page:

        http://sourceforge.net/projects/yuma/support

There are several sources of free information and tools for use with YANG and/or NETCONF.

The following section lists the resources available at this time.

1.2.1 WEB Sites

• •Netconf Central 

  • ◦http://www.netconfcentral.org/ 

  • ◦Yuma Home Page 

    • ▪Free information on NETCONF and YANG, tutorials, on-line YANG module validation and documentation database  

• •Yuma SourceFource OpenSource Project 

  • ◦http://sourceforge.net/projects/yuma/ 

    • ▪Download Yuma source and binaries; project forums and help 

• •Yang Central 

  • ◦http://www.yang-central.org 

  • ◦Free information and tutorials on YANG, free YANG tools for download 

• •NETCONF Working Group Wiki Page 

  • ◦http://trac.tools.ietf.org/wg/netconf/trac/wiki 

  • ◦Free information on NETCONF standardization activities and NETCONF implementations 

• •NETCONF WG Status Page 

  • ◦http://tools.ietf.org/wg/netconf/ 

  • ◦IETF Internet draft status for NETCONF documents 

• •libsmi Home Page 

  • ◦http://www.ibr.cs.tu-bs.de/projects/libsmi/ 

  • ◦Free tools such as smidump, to convert SMIv2 to YANG 

1.2.2 Mailing Lists

• •NETCONF Working Group 

  • ◦http://www.ietf.org/html.charters/netconf-charter.html 

  • ◦Technical issues related to the NETCONF protocol are discussed on the NETCONF WG mailing list.  Refer to the instructions on the WEB page for joining the mailing list. 

• •NETMOD Working Group 

  • ◦http://www.ietf.org/html.charters/netmod-charter.html 

  • ◦Technical issues related to the YANG language and YANG data types are discussed on the NETMOD WG mailing list.  Refer to the instructions on the WEB page for joining the mailing list. 

1.3 Conventions Used in this Document

The following formatting conventions are used throughout this document:

Documentation Conventions

2 Software Overview

2.1 Introduction

Refer to section 3 of the Yuma User Manual for a complete introduction to Yuma Tools.

This section focuses on the software development aspects of NETCONF, YANG, and the netconfd server.

2.1.1 Intended Audience

This document is intended for developers of server instrumentation library software, which can be used with the programs in the Yuma suite.  It covers the design and operation of the netconfd server, and the development of server instrumentation library code, intended for use with the netconfd server.

2.1.2 What does Yuma Do?

The Yuma Tools suite provides automated support for development and usage of network management information.  Refer to the Yuma User Guide for an introduction to the YANG data modeling language and the NETCONF protocol.

This section describes the Yuma development environment and the basic tasks that a software developer needs to perform, in order to integrate YANG module instrumentation into a device.

This manual contains the following information:

• •Yuma Development Environment 

• •Yuma Runtime Environment 

• •Yuma Source Code Overview 

• •Yuma Server Instrumentation Library Development Guide 

Yuma Tools programs are written in the C programming language, using the 'gnu99' C standard, and should be easily integrated into any operating system or embedded device that supports the Gnu C compiler.

2.1.3 What is a Yuma Root?

The Yuma Tools programs will search for some types of files in default locations

• •YANG Modules:  The 'modules' sub-directory is used as the root of the YANG module library. 

• •Client Scripts: The yangcli program looks in the 'scripts' sub-directory for user scripts. 

• •Program Data: The yangcli and netconfd programs look for saved data structures in the 'data' sub-directory. 

2.1.4 Searching Yuma Roots

1) $HOME Directory

The first Yuma root checked when searching for files is the directory identified by the $HOME environment variable.  If a '$HOME/modules', '$HOME/data'. and/or '$HOME/scripts' directory exists, then it will be checked for the specified file(s).

2) The $YUMA_HOME Directory

The second Yuma root checked when searching for files is the directory identified by the $YUMA_HOME environment variable.  This is usually set to private work directory, but a shared directory could be used as well.  If a '$YUMA_HOME/modules', '$YUMA_HOME/data'. and/or '$YUMA_HOME/scripts' directory exists, then it will be checked for the specified file(s).

3) The $YUMA_INSTALL Directory

The last Yuma root checked when searching for files is the directory identified by the $YUMA_INSTALL environment variable.  If it is not set, then the default value of '/usr/share/yuma' is used instead. This is usually set to the public directory where all users should find the default modules.  If a '$YUMA_INSTALL/modules', '$YUMA_INSTALL/data'. and/or '$YUMA_INSTALL/scripts' directory exists, then it will be checked for the specified file(s).

2.1.5 What is a SIL?

A SIL is a Server Instrumentation Library.  It contains the 'glue code' that binds YANG content (managed by the netconfd server), to your networking device, which implements the specific behavior, as defined by the YANG module statements.

The netconfd server handles all aspects of the NETCONF protocol operation, except data model semantics that are contained in description statements.  The server uses YANG files directly, loaded at boot-time or run-time, to manage all NETCONF content, operations, and notifications.

Callback functions are used to hook device and data model specific behavior to database objects and RPC operations.  The yangdump program is used to generate the initialization, cleanup, and 'empty' callback functions for a particular YANG module.  The callback functions are then completed (by you), as required by the YANG module semantics.  This code is then compiled as a shared library and made available to the netconfd server.  The 'load' command (via CLI, configuration file, protocol operation) is used (by the operator) to activate the YANG module and its SIL.

2.1.6 Auto-generated SIL Files

The SIL code for a YANG module can be generated with the make_sil_dir script described in the next section.  This script can generate 'combined' SIL files or 'split' SIL files (if the –split parameter is present).  

A split SIL module for foo.yang would be generated using the following files:

A combined SIL module for foo.yang would be generated using the following files:

2.1.7 Basic Development Steps

The steps needed to create server instrumentation for use within Yuma are as follows:

• •Create the YANG module data model definition, or use an existing YANG module. 

• • ◦Validate the YANG module with the yangdump program and make sure it does not contain any errors.  All warnings should also be examined to determine if they indicate data modeling bugs or not. 

• • ◦Example toaster.yang 

   

• •Make sure the $YUMA_HOME environment variable is defined, and pointing to your Yuma development tree. 

• •Create a SIL development subtree  

• • ◦Generate the directory structure and the Makefile with the make_sil_dir script, installed in the /usr/bin directory.  This step will also call yangdump to generate the initial H and C files file the SIL. 

     

  • ◦Example: mydir> make_sil_dir –split test 

   

• •Use your text editor to fill in the device-specific instrumentation for each object, RPC method, and notification.  (In this example, edit test/src/u_test.c)  Almost all possible NETCONF-specific code is either handled in the central stack, or generated automatically. so this code is responsible for implementing the semantics of the YANG data model. 

• •Compile your code 

  • ◦Use the 'make' command in the SIL 'src' directory.  This should generate a library file in the SIL 'lib' directory. 

  • ◦Example:  mydir/test/src> make 

   

• •Install the SIL library so it is available to the netconfd server. 

  • ◦Use the 'make install' command in the SIL 'src' directory. 

  • ◦Example:  mydir/test/src> sudo make install 

   

• •Run the netconfd server (or build it again if linking with static libraries) 

• •Load the new module 

  • ◦Be sure to add a 'load' command to the configuration file if the module should be loaded upon each reboot. 

  • ◦yangcli Example: load test 

   

• •The netconfd server will load the specified YANG module and the SIL and make it available to all sessions. 

2.2 Yuma Source Files

This section describes the files that are contained in the yuma-source package.

The important C include files are copied into /usr/include/yuma when the yuma-dev package is installed.  The full set of installation sources is installed in /usr/share/yuma/src, if the yuma-source package is installed. Yuma tools will check the $YUMA_HOME/src sub-tree before checking this default installation location.  This allows a working copy of the Yuma sources to be used instead of the installation copy, in case it has been modified for a particular embedded system (for example).

This section lists the files that are included within the netconf/src directory.

2.2.1 src/ncx Directory

This directory contains the code that is used to build the libncx.so binary shared library that is used by all Yuma Tools programs.  It handles many of the core NETCONF/YANG data structure support, including all of the YANG/YIN, XML,  and XPath processing.  The following table describes the purpose of each file.  Refer to the actual include file (e.g., ncx.h in /usr/include/yuma) for more details on each external function in each C source module.

src/ncx C Modules

2.2.2 src/platform Directory

This directory contains platform support include files and Makefile support files.  It is used by all Yuma C modules to provide an insulating layer between Yuma programs and the hardware platform that is used.  For example the m__getMem, m__getObj, and m__freeMem macros are used instead of malloc and free functions directly.

The following table describes the files that are contained in this directory:

src/platform Files

2.2.3 src/agt Directory

This directory contains the NETCONF server implementation and built-in module SIL code.  A static library called libagt.a is built and statically linked within the netconfd program.

The following table describes the C modules contained in this directory:

src/agt C Modules

2.2.4 src/mgr Directory

This module contains the NETCONF client support code.  It handles all the basic NETCONF details so a simple internal API can be used by NETCONF applications such as yangcli.   A static library called libmgr.a is built and statically linked within the yangcli program.

The following table describes the C modules contained in this directory:

src/mgr C Modules

2.2.5 src/subsys Directory

This directory contains the netconf-subsystem program.  This is a thin-client application that just transfers input and output between the SSH server and the NETCONF server.  It contains one C source module called netconf-subsystem.  This is a stand-alone binary that is part of the yuma-server package.  It is installed in the /usr/sbin/ directory.

2.2.6 src/netconfd Directory

This directory contains the netconfd program, which implements the NETCONF server. It contains one C module called netconfd, which defines the NETCONF server 'main' function.  This is a stand-alone binary that is part of the yuma-server package.  It is installed in the /usr/sbin/ directory.

2.2.7 src/yangcli Directory

This directory contains the yangcli program, which is the Yuma NETCONF client program.  This is a stand-alone binary that is part of the yuma-client package.  It is installed in the /usr/bin/ directory.

The following table describes the C modules contained in this directory:

src/yangcli C Modules

2.2.8 src/yangdiff Directory

This directory contains the yangdiff program, which is the Yuma YANG module compare program.  This is a stand-alone binary that is part of the yuma-client package.  It is installed in the /usr/bin/ directory.

The following table describes the C modules contained in this directory:

src/yangdiff

2.2.9 src/yangdump Directory

This directory contains the yangdump program, which is the Yuma YANG compiler program.  This is a stand-alone binary that is part of the yuma-client package.  It is installed in the /usr/bin/ directory.

The following table describes the C modules contained in this directory:

src/yangdump C Modules

2.3 Server Design

This section describes the basic design used in the netconfd server.

Initialization:

The netconfd server will process the YANG modules, CLI parameters, config file parameters, and startup device NETCONF database, then wait for NETCONF sessions.

ncxserver Loop:

The SSH2 server will listen for incoming connections which request the 'netconf' subsystem.

When a new session request is received, the netconf-subsystem program is called, which opens a local connection to the netconfd server, via the ncxserver loop.  NETCONF <rpc> requests are processed by the internal NETCONF stack.  The module-specific callback functions (blue boxes) can be loaded into the system at build-time or run-time.  This is the device instrumentation code, also called a server implementation library (SIL).  For example, for libtoaster, this is the code that controls the toaster hardware.

Cleanup:

If the <shutdown> or <reboot> operations are invoked, then the server will cleanup.  For a reboot, the init cycle is started again, instead of exiting the program.

2.3.1 YANG Native Operation