C++ and Lex/Yacc

This is some notes on lex and yacc. I think I have decent bit of knowledge in debugging linux, C/C++, lex/yacc and some related stuff, which I have accumulated over the years. I find myself telling people same stuff over and over. So this is an attempt to decrease amount of talking I do and yet share stuff with people.

For last three to four days, I have been playing with bison, bison++ and lex. The problem is like this.

I have two files, categories.lex, categories.yy. The categories.y file has some C++ code in it and my yyparse() will be called from C++ code in file test.C. If everything was in 'C' language, compilation would be straight forward. The problem is how do I compile each of these files, how do I call g++/gcc on the files so I can make a usable executable called "test". I have two bison's installed on my system. One is called "bison" which generates parser in 'C', another is called bison++ which generates parser in C++. See below for differences between bison and bison++

The solution is to export yyparse, yylex, yywrap, yyerror functions from *yy file using the phrase 'extern "C"'. Then .lex file which will be compiled with lex/gcc will get the function names in a unmangled manner.

So here are the commands I used and various error msgs I got during the process. just FYI, I have used all combinations of bison, bison++ and also tried to compile lex o/p with g++ as well as gcc. In the end, gcc for lex o/p succeeded.

$>lex -Pcategories categories.lex

This generated lex.categories.c file. Note that now yylex will be renamed to categorieslex because of the directive "-Pcategories"

$>gcc -c lex.categories.o
(note: Its GCC not G++)

$>bison -d -pcategories categories.yy

This generates file categories.tab.hh and categories.tab.cc. You must include categories.tab.hh in categories.lex. The main parse routine will now be categoriesparse() instead of yyprase (because of '-pcategories' directive).

$>g++ -c categories.tab.cc

Finally in test.C, add the following line
int categoriesparse();

You can all categoriesparse() from test.C just like anyother function.

$>g++ categories.tab.cc lex.categories.o test.o -o test

This should give you final output.

Using bison++ instead of bison in the command "bison -d -pcategories categories.yy" will generate two files categories.yy.tab.h and categories.yy.tab.c. A class of (default) name parse is created and yyparse is a member of this class. I donot really know how to use it with categories.lex file. Note that bison create a C++ file but doesnt create a class (typically named parse), where parsing routines are included. It generates non object oriented code. However, bison++ uses OOPS and creates a wrapper class for parsing.

Here are some errors I got and possible solutions to look for. Also, find my categories.lex and categories.y files attached(final working versions)

$>g++ -g -L/home/romit/java/contentanalyzer lex.categories.cc categories.tab.o test.C
lex.categories.o: In function `categorieslex':
lex.categories.c:(.text+0x227): undefined reference to `categorieslval'
categories.tab.o: In function `parse::parse()':
categories.tab.c:(.text+0x12): undefined reference to `vtable for parse'
categories.tab.o: In function `parse::parse()':
categories.tab.c:(.text+0x20): undefined reference to `vtable for parse'

Solution: You are using bison++ for categories.y. You may want to change it to bison. Also, you generated C++ code fromm categories.lex file. Generate 'C' code and use gcc to generate lex.categories.o file.

$>g++ -g -L/home/romit/java/contentanalyzer lex.categories.cc categories.tab.o test.C
categories.lex: In member function ‘virtual int categoriesFlexLexer::yylex()’:
categories.lex:11: error: ‘categorieslval’ was not declared in this scope
categories.lex:13: error: ‘STRING’ was not declared in this scope
categories.lex:17: error: ‘SEMICOLON’ was not declared in this scope
categories.lex:21: error: ‘COLON’ was not declared in this scope

You are using bison++ for categories.y and g++ for lex.categories.c
Instead use bison and gcc respectively. Explore generating lex code in 'c' instead of C++.

If you have any comments/clarifications, please post post them in comments sections and I will be happy to respond.

--Rags