Prelude is a general-purpose network intrusion detection system, written entirely from scratch, in C. Right now, it handles all of the TCP/IP stack over Ethernet. Prelude is divided in three parts:
The interesting part is plugins: you can write your own, dedicated to a special intrusion detection that you know of and share it with others. For example, bundled with the source is a plugin to detect SYN flooding attacks. There is more, and you can write more. Moreover, plugins are only called when necessary. For example, Prelude won't call the aforementioned plugin when it receives a TCP packet which hasn't got the SYN flag set, nor will it call it if it receives an ICMP packet.
This document will focus on Prelude's architecure. The different parts of Prelude will be discussed in the order in which they are enumerated above, and with a complete description of the API when it comes to writing plugins.
To quote Rik Van Riel in /usr/src/linux/Documentation/sysctl/README:
Which means: this information is provided as is, without any warranty of any kind, included and not limited to, blah, blah, blah. However, this documentation is written in the intent that it will be useful, so if you have any suggestions, things you don't understand, flames, etc, you can send them by email to Yoann.
But please no questions like "What's TCP?", "What's a SYN flooding?", etc. If you need such a tool, it means that you already know these things. If not, the source package includes a docs directory, which contains all relevant RFCs :)
Prelude uses the pcap library in order to extract frames from a network interface. Framess are extracted as is, which means that a frame contains all the protocol stack, from Ethernet's IEEE 802.3 to TCP, and the "useful" data itself.
Next comes the decoding of this frame, which means isolating the Ethernet leading and trailing bytes, then do the same for the following "higher-level" protocols. Two methods are used:
Hardcoded protocol decoding
Hardcoded into Prelude itself (for matters of speed) are all the protocols used in the TCP/IP stack:
When existing, the data contains all upper level protocols, such as FTP, SHH, DNS, etc. These are taken into account by the second method:
Protocol Plugins
Note: these are not implemented for now
These plugins will take care of the aforementioned not-yet-decoded data. The reason for them not being hardcoded is the huge number of protocols possible and available, and the fact that you may have written your own. You may then write a protocol plugin in order to debug it.
TODO: find an interface and document it here
Unlike all other existing network debugging tools, Prelude does take care of IP fragmentation. It works as follows: when Prelude receives a fragmented packet, it will add it to the defragmentation stack. It stores subsequent fragments of the same packet in the stack until it receives the last one. The original packet is then reassembled and passed on to the intrusion detection plugins.
Note that all packets, at a fragmented state *and* at a defragmented state will be analyzed for security problem. This allows for detection of IP fragmentation attacks, for example.
(Yoann) I firstly started to code a new IP defragmentation stack, but after some thinking, i thought it was better to use an existing / heavily tested / heavily debuged one So i replaced mine with the linux-kernel one which is already rock solid and secure, I've ported the linux kernel defragmentation stack to be used by prelude.
Prelude also knows of all documented options of the TCP and IP protocols. In case a packet containing options is received, Prelude compares the total lenght of the TCP (or IP) header in the current packet and the "normal" length of such a header (that is, a header without options). This gives:
Let's look at how it works, taking TCP as an example. You will have guessed that __tcp_hdr (resp. __ip_hdr) points to the actual start of the TCP (resp. IP) header into the packet.
The field th_off in the TCP header gives the total number of 32-bit words into the TCP header, including options. In order to know the total size of the header in bytes, the only thing to do is to multiply this number by 4.
By substracting the size of an option-less TCP header (sizeof(struct tcphdr)) to this number, we now know the total lenght in bytes of the options.
When packet slicing is done, Prelude submits the packets to the relevant plugins. As already stated in the introduction, only the relevant packets are sent for analysis to the relevant plugins.
Plugins are sent the whole packet, and they will be able to access directly to the part which interests them thanks to the slicing chore done by Prelude. From then on, each plugin will do its work.
In order for a plugin to operate, it has to subscribe to Prelude. For that matter, a subscription system has been created, which allows the plugin to say that it wants this sort of packet but not that one, etc. Let's look more closely at this subscription system:
At initialisation time, the plugin will tell Prelude about what kind of packet it exactly wants by sending it an array of subscribtion_t structures. The structure is defined as follows:
typedef struct {
proto_t type;
const char *pf_rule;
} subscribtion_t;
where type is the protocol and pf_rule (Packet Filter rule) is an optional PF rule which a packet needs to match. A PF rules looks much like a BPF rule; here is an valid PF rule :
(tcp_flags=syn and tcp_flags=ack) or (dst_port=21)
Which means: "I want packet with flags SYN and ACK turned on or packet with destination port 21."
I will not give a list of packet filter rules here cause they aren't standardized at the moment.
Let's take an example: the network scan detection plugin, provided with the source package. It needs to look at all incoming TCP and UDP packets. Here is how it will subscribe to Prelude:
static subscribtion_t subscribtion[] = {
{ p_tcp, NULL, "Tcp proto" },
{ p_udp, NULL, "Udp proto" },
{ p_end, NULL, NULL }
};
The only black magic here is how a plugin detects an attack. This is where you (the programmer) come into play. For our example, it works as follows:
Yoann...
If the plugin is sure that it detected an attack, it then reports it to Prelude, and this is what we'll have a look at right now.
Detection plugins are compiled as shared objects. They should have two globally declared (ie, not static) functions:
void plug_init(DetectPublic_t *plugin);
This function is called by Prelude at plugin initialization time. The DetectPlugin_t structure contains information that should be filled by this function:
typedef struct {
proto_t type;
const char *filter;
} subscribtion_t;
typedef struct {
char *author;
char *name;
char *desc;
subscribtion_t *subscribtion;
unsigned int max_warn;
unsigned int reset_warn;
} DetectPublic_t;
Here is a sample code for a plugin initialisation:
static subscribtion_t subscribtion[] = {
{ p_tcp, "tcp_flag=syn"},
{ p_udp, NULL },
{ p_end, NULL }
};
void plug_init(DetectPublic_t *plugin) {
plugin->name = "ScanDetect";
plugin->desc = "This plugin will detect TCP/UDP scanning and SYN flooding attempt.";
plugin->subscribtion = subscribtion;
}
void plug_run(const Packet_t *packet, int depth);
This function is called by Prelude when it has a packet for the plugin and wants to activate the plugin. packet is a pointer to an array of sliced network packets, as built by Prelude.
The Packet_t structure is the result which Prelude builds from the raw frames read from a network interface:
typedef enum { p_ip, p_udp, p_tcp, p_icmp, p_igmp, p_arp, p_rarp, p_ether, p_data, p_end } proto_t;
union proto_u {
struct ether_header *ether_hdr;
struct __iphdr *ip;
struct __tcphdr *tcp;
struct udphdr *udp_hdr;
struct icmp *icmp_hdr;
struct igmp *igmp_hdr;
struct arphdr *arp_hdr;
PktData_t *pkt_data;
};
typedef struct {
proto_t proto;
union proto_u p;
} Packet_t;
For example, if your plugin checks TCP headers, you can access all TCP header of the packet by packet[depth].p.tcp.
plug_run is therefore the heart of your plugin. Your mixture has to start from here :)
Here is a description of functions which Prelude (well, more accurately, libprelude) provides to plugins. We will look at them by category.
Note: the API is not definitive yet - watch for revisions of this document!
Report functions
void plugin_do_report(priority_t priority, const char *report, ...);
This is the function which a plugin needs to use in order to make a report. The report argument and all further arguments are as for printf and friends. The priority argument can be one of p_low, p_medium and p_high.
This function is for a "one time warning", that is, a plugin will only use this function if it is able to detect an attack in just one packet.The priority argument can be one of p_low, p_medium or p_high.
void plugin_do_report_self(priority_t priority, Packet_t *packet, const char *report, ...);
This function is used by plugins which need to keep trace of several packets in order to detect an attack. An example is the scan detection plugin. The priority argument is as specified above.
Note that the function simply passes back a pointer to the packet, the plugin should therefore use the Pkt{Alloc,Copy,Free} functions described below if you need to keep track of one or more packet(s).
Packet manipulation functions If there is a function that match your need, you *should* use the provided one, and not bother to recode it. libprelude is there because it act as a cache between function called by plugins each time a packet is received... If three plugins call the same libprelude function, exemple : prelude_GetDepth(), prelude_GetDepth will be really used the first time it is called, but the second and third time it will just return the same result.
int prelude_GetDepth(const Packet_t *packet, int remove_me, proto_t proto);
Note: this is a bit clunky, and will be corrected soon. Remember the plug_run function above? You've been passed a depth argument. This is what you'll have to pass back as the remove_me argument here. It's going to be removed in the very near future (probably even before this doc has completed). When Prelude sends a packet to a plugin, it sends a Packet_t array of pointers. The Packet_t structure is as follows:
union proto_u {
struct ether_header *ether_hdr;
struct __iphdr *ip;
struct __tcphdr *tcp;
struct udphdr *udp_hdr;
struct icmp *icmp_hdr;
struct igmp *igmp_hdr;
struct arphdr *arp_hdr;
PktData_t *pkt_data;
};
typedef enum { p_ip, p_udp, p_tcp, p_icmp, p_igmp, p_arp, p_rarp, p_ether,
p_data, p_end } proto_t;
typedef struct {
proto_t proto;
union proto_u p;
} Packet_t;
The plugin is passed the whole packet, and each Packet_t element
points to a given protocol. You should pass the protocol you want as the
proto argument. You will then be returned the index at which you can
find what you want. -1 is returned on error. Provided that you want to have a
look at the TCP header, a sample code would be:
__tcphdr *my_header;
ret=preludeGetDepth(my_packet,remove_me,p-tcp);
if(ret == -1){
/*
* mixture;
*/
}
my_header=packet[ret].p.tcp;
/*
* More mixture;
*/
const char *prelude_GetCleanData(const char *data, size_t data_len);
This function is used in order to get printable data from a packet data. You may want this in order to regenerate a report and your report needs to dump it.
Packet copy functions
As mentioned above, you *should* use these functions if you need to keep track of old packets. The reason for this is that Prelude keeps a stack of all packets, stack which you won't access with the normal malloc() and friends. You don't want it, because the stack makes things faster - and simpler. For one, it strips down all bugs due to bad memory allocations by plugins ;)
Packet_t *prelude_PktAlloc(int depth); This function will allocate a Packet_t structure into Prelude's stack and return a pointer to it.
TODO: whats the depth argument for? void prelude_PktCpy(Packet_t *dst, const Packet_t *src, int depth); Like memcpy :-)
TODO: whats the depth argument for? void prelude_PktFree(Packet_t *packet); Frees a packet allocated with prelude_PktAlloc().
TCP and IP options parsing
TODO: list of possible TCP and IP options? In which way are they available to plugins?
TCP and IP option parsing is done by Prelude. The set of functions presented in this section helps telling Prelude what to do. All you have to do in order to {scan,mess with} TCP or IP options is the following declaration:
pktopt_t *whatever; /* a more talkative name is of course better */
As far as writing plugins is concerned, pktopt_t is and should remain a black box, so its structure will not be shown here. Let Prelude do its black magic and use the functions below :)
pktopt_t *prelude_InitIpOptParsing(struct __iphdr *ip);
pktopt_t *prelude_InitTcpOptParsing(struct __iphdr *ip);
These functions must be called in order to initialise the pktopt_t structure. Prelude will make it point to the correct place in the IP (or TCP) header, and will reuse it afterwards.
int prelude_GetNextIpOption(pktopt_t *option);
int prelude_GetNextTcpOption(pktopt_t *option);
These are the main functions for parsing options: you may call them with the pktopt_t which you initialised with the functions above. These functions do a one-by-one option parsing. If you are looking for an option in perticular, look at the functions below.
If these functions are called and there isn't any option left, they will return -1. Otherwise, the value of the option is returned.
int prelude_SearchIpOption(pktopt_t *option, int wanted);
int prelude_SearchTcopOption(pktopt_t *option, int wanted);
You should call this function if you're looking for a specific option. Similarly, the result is the desired option if found, -1 if not.
int prelude_GetNextIpOptionValue(pktopt_t *option);
int prelude_GetNextTcpOptionValue(pktopt_t *option);
If you are interested, not only in a TCP or IP option, but also by its value, here is the functions you will use. You should call them immediately after the corresponding GetNext or Searchfunction, of course.
Another point is that some options embed more than one value. This is the case for source routing in IP, for example.In this case, you may call this function recursively until you encounter value of -1, in which case it's high time you jumped to the next one :)
When Prelude receives an attack report from a plugin, the first thing it does is generate a report. The report consists of informations on the culprit packet(s), plus what the plugin says.
TODO: describe functions used for that
Prelude then translates the report into XDR format (eXternal Data Representation). This protocol, created by Sun for NFS, provides C programmers with routines to describe arbitrary data structures in a machine-independent fashion.
It then sends the XDR-encoded data to PreludeReport, which is reachable via a network socket. If the report server is unreachable, ie the socket has been closed, Prelude will save the report to a file, and resend it later. Reports are not kept into memory.
PreludeReport is the program in charge of formatting and forwarding all reports which it has received. It can receive reports from the local machine or from distant machines. Report formatting and forwarding is taken care of by another system of modules: ReportPlugins.
PreludeReport listens to two sockets: a Unix one (for local connections) and a TCP one (for remote connections). The port for TCP is 4000 for now, but it may change in the future. For TCP connections, PreludeReport uses the tcp-wrappers package in order to check whether it should accept or deny the connection. So, do not forget to fill your /etc/hosts.{deny,allow} appropriately... Scheduled is also the use of an authentication scheme, but the method isn't defined at the moment. Maybe it will be RSA or Kerberos.
When PreludeReport receives a report, it decodes the data (don't forget that it has been encoded with XDR) and passes it to all currently loaded ReportPlugins. These plugins will then issue human readable reports. For example, you can have a plugin mail the report to a given address, write the report to the system logs (using syslogd for example), format it in HTML and so on. As for detectin plugins, you can write one that fits exactly your needs.
This is only a not-so-distant project for Prelude. In FW, understand FireWall.
As of Prelude Report, this server can listen on a Unix or a TCP socket. This port is not defined. Yet. It will also use the same host verification/authentication scheme which PreludeReport uses.
The role of Prelude Dynamic FW will be to acknowledge requests from PreludeReport in case a plugin reports that a given host is trying to mess up with the system. Prelude Dynamic FW will then simply disallow connections from the given host, using firewalling capabilities of the host it runs on.