mc/vfs/samba/parsing.doc

Chris Hertel, Samba Team
November 1997

This is a quick overview of the lexical analysis, syntax, and semantics
of the smb.conf file.

Lexical Analysis:

  Basically, the file is processed on a line by line basis.  There are
  four types of lines that are recognized by the lexical analyzer
  (params.c):

  Blank lines           - Lines containing only whitespace.
  Comment lines         - Lines beginning with either a semi-colon or a
                          pound sign (';' or '#').
  Section header lines  - Lines beginning with an open square bracket
                          ('[').
  Parameter lines       - Lines beginning with any other character.
                          (The default line type.)

  The first two are handled exclusively by the lexical analyzer, which
  ignores them.  The latter two line types are scanned for

  - Section names
  - Parameter names
  - Parameter values

  These are the only tokens passed to the parameter loader
  (loadparm.c).  Parameter names and values are divided from one
  another by an equal sign: '='.


  Handling of Whitespace:

  Whitespace is defined as all characters recognized by the isspace()
  function (see ctype(3C)) except for the newline character ('\n')
  The newline is excluded because it identifies the end of the line.

  - The lexical analyzer scans past white space at the beginning of a
    line.

  - Section and parameter names may contain internal white space.  All
    whitespace within a name is compressed to a single space character. 

  - Internal whitespace within a parameter value is kept verbatim with
    the exception of carriage return characters ('\r'), all of which
    are removed.

  - Leading and trailing whitespace is removed from names and values.


  Handling of Line Continuation:

  Long section header and parameter lines may be extended across
  multiple lines by use of the backslash character ('\\').  Line
  continuation is ignored for blank and comment lines.

  If the last (non-whitespace) character within a section header or on
  a parameter line is a backslash, then the next line will be
  (logically) concatonated with the current line by the lexical
  analyzer.  For example:

    param name = parameter value string \
    with line continuation.

  Would be read as

    param name = parameter value string     with line continuation.

  Note that there are five spaces following the word 'string',
  representing the one space between 'string' and '\\' in the top
  line, plus the four preceeding the word 'with' in the second line.
  (Yes, I'm counting the indentation.)

  Line continuation characters are ignored on blank lines and at the end
  of comments.  They are *only* recognized within section and parameter
  lines.


  Line Continuation Quirks:
  
  Note the following example:

    param name = parameter value string \
    \
    with line continuation.

  The middle line is *not* parsed as a blank line because it is first
  concatonated with the top line.  The result is

    param name = parameter value string         with line continuation.

  The same is true for comment lines.

    param name = parameter value string \
    ; comment \
    with a comment.

  This becomes:
  
    param name = parameter value string     ; comment     with a comment.

  On a section header line, the closing bracket (']') is considered a
  terminating character, and the rest of the line is ignored.  The lines
  
    [ section   name ] garbage \
    param  name  = value

  are read as

    [section name]
    param name = value



Syntax:

  The syntax of the smb.conf file is as follows:

  <file>            :==  { <section> } EOF

  <section>         :==  <section header> { <parameter line> }

  <section header>  :==  '[' NAME ']'

  <parameter line>  :==  NAME '=' VALUE NL


  Basically, this means that
  
    - a file is made up of zero or more sections, and is terminated by
      an EOF (we knew that).

    - A section is made up of a section header followed by zero or more
      parameter lines.

    - A section header is identified by an opening bracket and
      terminated by the closing bracket.  The enclosed NAME identifies
      the section.

    - A parameter line is divided into a NAME and a VALUE.  The *first*
      equal sign on the line separates the NAME from the VALUE.  The
      VALUE is terminated by a newline character (NL = '\n').


About params.c:

  The parsing of the config file is a bit unusual if you are used to
  lex, yacc, bison, etc.  Both lexical analysis (scanning) and parsing
  are performed by params.c.  Values are loaded via callbacks to
  loadparm.c.

--------------------------------------------------------------------------

                                  Samba DEBUG
                                       
Chris Hertel, Samba Team
July, 1998
   
   Here's the scoop on the update to the DEBUG() system.
   
   First, my goals are:
     * Backward compatibility (ie., I don't want to break any Samba code
       that already works).
     * Debug output should be timestamped and easy to read (format-wise).
     * Debug output should be parsable by software.
     * There should be convenient tools for composing debug messages.
       
   NOTE: the Debug functionality has been moved from util.c to the new
   debug.c module.
   
New Output Syntax

   The syntax of a debugging log file is represented as:
  <debugfile> :== { <debugmsg> }

  <debugmsg>  :== <debughdr> '\n' <debugtext>

  <debughdr>  :== '[' TIME ',' LEVEL ']' FILE ':' [FUNCTION] '(' LINE ')'

  <debugtext> :== { <debugline> }

  <debugline> :== TEXT '\n'

   TEXT is a string of characters excluding the newline character.
   LEVEL is the DEBUG level of the message (an integer in the range
   0..10).
   TIME is a timestamp.
   FILE is the name of the file from which the debug message was
   generated.
   FUNCTION is the function from which the debug message was generated.
   LINE is the line number of the debug statement that generated the
   message.
   
   Basically, what that all means is:
     * A debugging log file is made up of debug messages.
     * Each debug message is made up of a header and text. The header is
       separated from the text by a newline.
     * The header begins with the timestamp and debug level of the
       message enclosed in brackets. The filename, function, and line
       number at which the message was generated follow. The filename is
       terminated by a colon, and the function name is terminated by the
       parenthesis which contain the line number. Depending upon the
       compiler, the function name may be missing (it is generated by the
       __FUNCTION__ macro, which is not universally implemented, dangit).
     * The message text is made up of zero or more lines, each terminated
       by a newline.
       
   Here's some example output:

    [1998/08/03 12:55:25, 1] nmbd.c:(659)
      Netbios nameserver version 1.9.19-prealpha started.
      Copyright Andrew Tridgell 1994-1997
    [1998/08/03 12:55:25, 3] loadparm.c:(763)
      Initializing global parameters

   Note that in the above example the function names are not listed on
   the header line. That's because the example above was generated on an
   SGI Indy, and the SGI compiler doesn't support the __FUNCTION__ macro.
   
The DEBUG() Macro

   Use of the DEBUG() macro is unchanged. DEBUG() takes two parameters.
   The first is the message level, the second is the body of a function
   call to the Debug1() function.
   
   That's confusing.
   
   Here's an example which may help a bit. If you would write

     printf( "This is a %s message.\n", "debug" );

   to send the output to stdout, then you would write

     DEBUG( 0, ( "This is a %s message.\n", "debug" ) );

   to send the output to the debug file.  All of the normal printf()
   formatting escapes work.
   
   Note that in the above example the DEBUG message level is set to 0.
   Messages at level 0 always print.  Basically, if the message level is
   less than or equal to the global value DEBUGLEVEL, then the DEBUG
   statement is processed.
   
   The output of the above example would be something like:

    [1998/07/30 16:00:51, 0] file.c:function(128)
      This is a debug message.

   Each call to DEBUG() creates a new header *unless* the output produced
   by the previous call to DEBUG() did not end with a '\n'. Output to the
   debug file is passed through a formatting buffer which is flushed
   every time a newline is encountered. If the buffer is not empty when
   DEBUG() is called, the new input is simply appended.

   ...but that's really just a Kludge. It was put in place because
   DEBUG() has been used to write partial lines. Here's a simple (dumb)
   example of the kind of thing I'm talking about:

    DEBUG( 0, ("The test returned " ) );
    if( test() )
      DEBUG(0, ("True") );
    else
      DEBUG(0, ("False") );
    DEBUG(0, (".\n") );

   Without the format buffer, the output (assuming test() returned true)
   would look like this:

    [1998/07/30 16:00:51, 0] file.c:function(256)
      The test returned
    [1998/07/30 16:00:51, 0] file.c:function(258)
      True
    [1998/07/30 16:00:51, 0] file.c:function(261)
      .

   Which isn't much use. The format buffer kludge fixes this problem.
   
The DEBUGADD() Macro

   In addition to the kludgey solution to the broken line problem
   described above, there is a clean solution. The DEBUGADD() macro never
   generates a header. It will append new text to the current debug
   message even if the format buffer is empty. The syntax of the
   DEBUGADD() macro is the same as that of the DEBUG() macro.

    DEBUG( 0, ("This is the first line.\n" ) );
    DEBUGADD( 0, ("This is the second line.\nThis is the third line.\n" ) );

   Produces
    [1998/07/30 16:00:51, 0] file.c:function(512)
      This is the first line.
      This is the second line.
      This is the third line.

The DEBUGLVL() Macro

   One of the problems with the DEBUG() macro was that DEBUG() lines
   tended to get a bit long. Consider this example from
   nmbd_sendannounce.c:

  DEBUG(3,("send_local_master_announcement: type %x for name %s on subnet %s for workgroup %s\n",
            type, global_myname, subrec->subnet_name, work->work_group));

   One solution to this is to break it down using DEBUG() and DEBUGADD(),
   as follows:

  DEBUG( 3, ( "send_local_master_announcement: " ) );
  DEBUGADD( 3, ( "type %x for name %s ", type, global_myname ) );
  DEBUGADD( 3, ( "on subnet %s ", subrec->subnet_name ) );
  DEBUGADD( 3, ( "for workgroup %s\n", work->work_group ) );

   A similar, but arguably nicer approach is to use the DEBUGLVL() macro.
   This macro returns True if the message level is less than or equal to
   the global DEBUGLEVEL value, so:

  if( DEBUGLVL( 3 ) )
    {
    dbgtext( "send_local_master_announcement: " );
    dbgtext( "type %x for name %s ", type, global_myname );
    dbgtext( "on subnet %s ", subrec->subnet_name );
    dbgtext( "for workgroup %s\n", work->work_group );
    }

   (The dbgtext() function is explained below.)
   
   There are a few advantages to this scheme:
     * The test is performed only once.
     * You can allocate variables off of the stack that will only be used
       within the DEBUGLVL() block.
     * Processing that is only relevant to debug output can be contained
       within the DEBUGLVL() block.
       
New Functions

   dbgtext()
          This function prints debug message text to the debug file (and
          possibly to syslog) via the format buffer. The function uses a
          variable argument list just like printf() or Debug1(). The
          input is printed into a buffer using the vslprintf() function,
          and then passed to format_debug_text().
          
          If you use DEBUGLVL() you will probably print the body of the
          message using dbgtext(). 
          
   dbghdr()
          This is the function that writes a debug message header.
          Headers are not processed via the format buffer. Also note that
          if the format buffer is not empty, a call to dbghdr() will not
          produce any output. See the comments in dbghdr() for more info.
          
          It is not likely that this function will be called directly. It
          is used by DEBUG() and DEBUGADD().
          
   format_debug_text()
          This is a static function in debug.c. It stores the output text
          for the body of the message in a buffer until it encounters a
          newline. When the newline character is found, the buffer is
          written to the debug file via the Debug1() function, and the
          buffer is reset. This allows us to add the indentation at the
          beginning of each line of the message body, and also ensures
          that the output is written a line at a time (which cleans up
          syslog output).