[Click] FileReader and Pattern Matching with Click

[Neel Sheyal]

Hi Beyers -

1. Do you guys have any recommendation for a regex matching library in C++
(used with click)? I am using RE2( https://code.google.com/p/re2) but if
you guys (or anyone for that matter) has used one with good performance, I
would appreciate sharing the information.

2. Also, I am converting the payload to a String using t he _c.str()
function. Since I need to do pattern matching, I figured I would need the
String object. Is there other (and better) ways of doing pattern matching
on the payload data?

Thanks,
Neel



On Sat, May 11, 2013 at 3:41 AM, Beyers Cronje <bcronje at gmail.com> wrote:

> Hi Neel,
>
> 1. I would think you should do UDP encap first and then Ether encap:
> FromSyslogFile -> UDPIPEncap -> EtherEncap. Personally I would use Socket
> http://read.cs.ucla.edu/click/elements/socket to transmit the packets,
> then
> you dont need to worry about Ether/UDP encapsulation.
>
> 2. That sounds about right. Pattern matching can obviously be a performance
> bottle neck, so you might need to consider using multiple threads to do
> concurrent matching, e.g.
>
> Syslogdata -> q::ThreadSafeQueue;
> q -> sc1::StringClassifier
> q -> sc2:: StringClassifier
> q -> sc3:: StringClassifier
> etc
>
> You would then implement StringClassifier as a Pull element with a task or
> use Unqueue element and use StaticThreadSched to schedule each instance of
> it on its own thread.
>
> Beyers
>
>
> On Sat, May 11, 2013 at 6:23 AM, Neel Sheyal <latencybuster at gmail.com
> >wrote:
>
> > Hi All,
> >
> >       I want to use click to create a distributed pattern matching and
> > messaging system.
> >
> > The starting point is a series of Syslog messages. Since Click does not
> > have an inbuilt Syslog (AFAIK) element, I am dumping all the Syslog
> > messages to a file index first. Then, I am asking Click to read each line
> > of the Syslog content, convert it to a packet that passes through my
> > PatternMatcher element . Here is what I am trying to do and failing:
> >
> >
> > 1. FromSyslogFile(<filename>) -> EtherEncap(..) -> UDPIPEncap(..) ->
> > ToDump()
> > EherEncap is correctly putting the Ethernet header but UDPIPEncap() is
> not
> > doing anything.. How do I put Ethernet/IP/UDP header to my data that I am
> > reading from my file using the element (created by me) FromSyslogFile()?
> >
> > 2. Suppose Step 1 succeeds i.e. I have the packet with the Syslog data in
> > the payload, I am now want it to pass through my element,
> > StringClassifer(<regex1>, <regex2>,..). I am using the C++ lib re2(
> > https://code.google.com/p/re2/) inside my StringClassifier class for
> > pattern matching. Is this the correct/recommended approach to pattern
> > matching at  Layer7?
> >
> > Thanks,
> > NeelSheyal
> > _______________________________________________
> > click mailing list
> > click at amsterdam.lcs.mit.edu
> > https://amsterdam.lcs.mit.edu/mailman/listinfo/click
> >
> _______________________________________________
> click mailing list
> click at amsterdam.lcs.mit.edu
> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>