Monday, November 19, 2007

Busy, busy, busy

Well between work, soccer, and programming in my spare time, I haven't updated this in a while. So what's the latest scoop in VCF land? Well while the release isn't yet ready (yes I know, it's been a ridiculously long time between releases, but between personnel changes, and volunteer work on Zod's campaign...), there are some pretty cool things in the pipeline.

Most of the newly added threading code, such as thread methods/functions, thread pools, and the revamped Delegate classes have been added and incorporated into the rest of the VCF kits. We have added support for regular expressions, java script, and network classes back into the trunk (these were in the bluesky directory).

Coming next is advanced support for XML. Not just simple parsing anymore, as we already have that right now, but full blown support for XML including XPath and XSLT. Support for a TextReader interface is also present, as well as getting the full XML document tree. You can also use delegates to handle parsing notification ala SAX2. This is all based on the libxml and lilbxslt libraries from xmlsoft.org. Here are some quick and dirty examples:

SAX2 parsing notification:



void MystartElementSAXFunc2( const xmlChar * name,
const xmlChar ** atts)
{
printf( "@MystartElementSAXFunc2 name: %s\n", (const char*) name );

if ( NULL != atts ) {
const xmlChar** tmp = atts;
while ( *tmp != 0 ) {
printf( "attr: %s\n", (const char*)*tmp );
tmp ++;
}
}
}

#define testxml "<stuff a=\"1\" b=\"23\">"\
"Wow!"\
"<animal>Zebra</animal>"\
"<animal>Cow</animal>"\
"</stuff>"


int main( int argc, char** argv ){

FoundationKit::init( argc, argv );
XMLKit::init( argc, argv );

System::println( "XMLkit XML version: " + XMLKit::getXMLVersion() );
System::println( "XMLkit XSLT version: " + XMLKit::getXSLTVersion() );


{

XMLSaxParser p;
p.StartElement += MystartElementSAXFunc2;

p.parse( testxml );
}
}


A simple TextReader example:

int main( int argc, char** argv ){

FoundationKit::init( argc, argv );
XMLKit::init( argc, argv );
XMLTextReader rdr;
rdr.setXML( testxml );

System::println( "lang: " + rdr.getLang() );
System::println( "uri: " + rdr.getBaseURI() );
System::println( "ns: " + rdr.getNamespaceURI() );

while ( rdr.read() ) {
System::println( "Name: " + rdr.getName() +
" depth: " + rdr.getCurrentDepth() );
System::println( "Inner XML: { " + rdr.readInnerXml() + " }" );
}
}


While this happened the Internet kit got support for asynchronous url retrieval. Before you get get the contents of a URL, but it was blocking. Now a new class has been added so that you can get the contents in async mode. You can do things like this:


AsyncURL* url = new AsyncURL("http://www.w3schools.com/xpath/books.xml");

And in some other thread you can wait for the url to finish:

url->wait();
//do something with the URL contents.
delete url;


Or you can be notified via the delegate:



void urlCompleted( URLEvent* event )
{
URL* url = (URL*)event->getSource();

//do something with the url contents

url->free();
}

AsyncURL* url = new AsyncURL("http://www.w3schools.com/xpath/books.xml");
url->DataComplete += urlCompleted;
url->get();


Finally you can simply wait and block till it's done and extract the contents as a string:

AsyncURL url("http://www.w3schools.com/xpath/books.xml");
url.get();
url.wait();
String xml = url.getDataAsString();


You can create a xml document from the xml string:

AsyncURL url("http://www.w3schools.com/xpath/books.xml");
url.get();
url.wait();
String xml = url.getDataAsString();
XmlDocument doc;
doc.setXML(xml);


You can then iterate through the child nodes:

XmlNode root = doc.getRoot();
std::vector children;
root.getChildren( children );
std::vector::iterator cit = children.begin();
while ( cit != children.end() ) {
XmlNode& child = *cit;
System::println( "Name: " + child.getName() +
", path: " + child.getPath() );
++cit;
}


You can use XPath to perform queries:

std::vector nodes;
VariantData res = doc.matches( "/bookstore/book[1]", nodes );
System::println( "doc.matches() returned: " + res.toString() );

for (size_t i=0;i < nodes.size();i++ ) {
const XmlNode& n = nodes[i];
System::println( "Node name: " +
n.getName() + " path: " +
n.getPath() + " text: " +
n.getContent() );
}


The result may be a calculation:

std::vector nodes;
VariantData res = doc.matches( "sum(//price/text())", nodes );
System::println( "doc.matches() returned: " + res.toString() );
}


There's more as well, you can see the complete file here.

In addition to XML there's been some initial work done with creating a cryptography kit based on the OpenSSL library, providing support for most of the core sections of the library. Currently we have support for ciphers, hashes, ASN1, private/public keys, certificates (x509), base64 encoding/decoding, random numbers/data, Big Integers (what OpenSSL refers to as the BIGNUM struct), and IO. Now that those are wrapped up, I'm looking into the SSL side of things. What follows are some samples of the current code:

Hashes:


using namespace VCF::Crypto;

MD2 md2;
MD5 md5;
VCF::Crypto::SHA1 sha1;
VCF::Crypto::SHA sha;
DSS dss;
DSS1 dss1;
RipeMD160 rp160;
char mess1[] = "Test Message";
char mess2[] = "Hello World\n";
MessageDigest::DigestResult res =
md5.hash( (const unsigned char*)mess1, strlen(mess1) );

int i = 0;

printf("md5 Digest is (%d bytes): ", md5.size() );
for(i = 0; i < res.size(); i++) {
printf("%X", res[i]);
}
printf("\n");

Repeated as necessary for the other hashes.

Big integers are pretty simple to use:

BigInteger bi;
bi = 1;

BigInteger bi2 = -120;
int j = bi2;

bi = 1312;
bi2 *= bi;

String bs = bi2.toString();


Generate an RSA key pair and write it to disk:

RSAKeyPair kp;
kp.generate( 2048, RSAKeyPair::expRSA_F4 );
RSAPrivateKey privk = kp.getPrivateKey();
RSAPublicKey pubk = kp.getPublicKey();

privk.setPassword( PASSWORD );

DataEncryptionStandard3CBC des3cbc;
privk.setEncryptionCipher( &des3cbc );


FileOutputStream fos("test2-priv.pem");
fos.write( &privk );

fos.close();

fos.open( "test2-pub.pem" );
fos.write( &pubk );



You can read in the private key buy supplying a password:

RSAPrivateKey pk;
pk.setPassword( PASSWORD );
FileInputStream fis("test2-priv.pem");
fis.read( &pk );

or by using a calback function to supply the password string:

String gimmeThePassword()
{
return String(PASSWORD);
}


RSAPrivateKey pk;
pk.PasswordPrompt += gimmeThePassword;

FileInputStream fis("test2-priv.pem");
fis.read( &pk );

System::println( "RSAPrivateKey: \n" + pk );


X509 certificates can be generated

X509Certificate c;
Key k = c.generateRSA(512,0,365,2);
Key::KeyType t = k.getType();

or read in from an input stream:

FileInputStream fis("test509.cer");
PEMInputStream pis(&fis);

X509Certificate* c2 = pis.readCertificate("");
delete c2;


Obviously more work needs to be done here, but I think this is a good start. The next big thing is understanding how to make use of SSL and how to integrate that.

Finally there's been some progress on writing a ZIP input/output stream class to handle compression/decompression with ZLib. Not so much to post about on that so far.

Well at least everyone knows know I haven't completely thrown in the towel! :)