[Click] FileReader and Pattern Matching with Click

[Beyers Cronje]

Hi Neel,

1. I would think you should do UDP encap first and then Ether encap:
FromSyslogFile -> UDPIPEncap -> EtherEncap. Personally I would use Socket
http://read.cs.ucla.edu/click/elements/socket to transmit the packets, then
you dont need to worry about Ether/UDP encapsulation.

2. That sounds about right. Pattern matching can obviously be a performance
bottle neck, so you might need to consider using multiple threads to do
concurrent matching, e.g.

Syslogdata -> q::ThreadSafeQueue;
q -> sc1::StringClassifier
q -> sc2:: StringClassifier
q -> sc3:: StringClassifier
etc

You would then implement StringClassifier as a Pull element with a task or
use Unqueue element and use StaticThreadSched to schedule each instance of
it on its own thread.

Beyers


On Sat, May 11, 2013 at 6:23 AM, Neel Sheyal <latencybuster at gmail.com>wrote:

> Hi All,
>
>       I want to use click to create a distributed pattern matching and
> messaging system.
>
> The starting point is a series of Syslog messages. Since Click does not
> have an inbuilt Syslog (AFAIK) element, I am dumping all the Syslog
> messages to a file index first. Then, I am asking Click to read each line
> of the Syslog content, convert it to a packet that passes through my
> PatternMatcher element . Here is what I am trying to do and failing:
>
>
> 1. FromSyslogFile(<filename>) -> EtherEncap(..) -> UDPIPEncap(..) ->
> ToDump()
> EherEncap is correctly putting the Ethernet header but UDPIPEncap() is not
> doing anything.. How do I put Ethernet/IP/UDP header to my data that I am
> reading from my file using the element (created by me) FromSyslogFile()?
>
> 2. Suppose Step 1 succeeds i.e. I have the packet with the Syslog data in
> the payload, I am now want it to pass through my element,
> StringClassifer(<regex1>, <regex2>,..). I am using the C++ lib re2(
> https://code.google.com/p/re2/) inside my StringClassifier class for
> pattern matching. Is this the correct/recommended approach to pattern
> matching at  Layer7?
>
> Thanks,
> NeelSheyal
> _______________________________________________
> click mailing list
> click at amsterdam.lcs.mit.edu
> https://amsterdam.lcs.mit.edu/mailman/listinfo/click
>