FLAIM (Framework for Log Anonymization and Information Management) is a modular tool designed to allow computer and network log sharing through application of complex data sanitization policies.[1]
FLAIM is aimed at 3 different user communities. First, FLAIM can be used by the security engineer who is investigating a broad incident spanning multiple organizations. Because of the sensitivity inherent in security relevant logs, many organizations are reluctant to share them. However, this reluctance inhibits the sharing necessary to investigate intrusions that commonly span organizational boundaries. Second, anyone designing log analysis or computer forensics tools needs data with which they can test their tools.[2] The larger and more diverse the data set, the more robust they can make their tools. For many, this means they must gather many logs from outside sources, not just what they can generate in-house. Again, this requires log sharing. Third, researchers in many computer science disciplines (e.g., network measurements, computer security, etc.) need large and diverse data sets to study. Having data sanitization tools available makes organizations more willing to share with these researchers their own logs.
While FLAIM is not the only log anonymizer, it is unique in its flexibility to create complex XML policies and its support for multiple log types.[1] More specifically, it is the only such tool to meet the following 4 goals. (1) FLAIM provides a diverse set of anonymization primitives. (2) FLAIM supports multiple log type, including linux process accounting logs, netfilter alerts, tcpdump traces and NFDUMP NetFlows.[4] (3) With a flexible anonymization policy language, complex policies that make trade-offs between information loss and security can be made. (4) FLAIM is modular and easily extensible to new types of logs and data. The anonymization engine is agnostic to the syntax of the actual log.
History
Work on log anonymization began in 2004 at the NCSA. At first this was for anonymizing logs in-house to share with the SIFT group. Soon there was a need for more powerful anonymization and anonymization of different types of logs.[5] CANINE was created to anonymize and convert between multiple formats of NetFlows.[6][7] This was a Java GUI-based tool. Later, Scrub-PA was created to anonymize Process Accounting logs.[8] Scrub-PA was based on the Java code used for CANINE. The development of both of these tools were funded under the Office of Naval Research NCASSR research center through the SLAGEL project.[9]
It was quickly realized that building one-off tools for each new log format was not the way to go. Also, the earlier tools were limited in that they could not be scripted from the command line. It was decided that a new, modular command line-based UNIX tool was needed. Because speed was also a concern, this tool need to be written in C++. With the successful acquisition of a Cyber Trust grant from the National Science Foundation, the LAIM Working Group was formed at the NCSA.[10] From this project headed by the PI, Adam Slagell, FLAIM was developed to overcome these limitations of CANINE and Scrub-PA. The first public version of FLAIM, 0.4., was released on July 23, 2006.[11]
^ abSlagell, A., Lakkaraju, K., and Luo, K., "FLAIM: A Multi-level Anonymization Framework for Computer and Network Logs," 20th USENIX Large Installation System Administration Conference (LISA '06), Washington, D.C., Dec., 2006.
^Slagell, A., Li, Y., and Luo, K., "Sharing Network Logs for Computer Forensics: A New Tool for the Anonymization of NetFlow Records," Computer Network Forensics Research (CNFR) Workshop, Athens, Greece, Sep., 2005.
^Luo, K., Li, Y., Slagell, A., and Yurcik, W., "CANINE: A NetFlow Converter/Anonymizer Tool for Format Interoperability and Secure Sharing," FLOCON — Network Flow Analysis Workshop, Pittsburgh, PA, Sep., 2005.
^Li, Y., Slagell, A., Luo, K., and Yurcik, W., "CANINE: A Combined Conversion and Anonymization Tool for Processing NetFlows for Security," 10th International Conference on Telecommunication Systems, Modeling and Analysis, Dallas, TX, Nov., 2005.
^Luo, K., Li, Y., Ermopoulos, C., Yurcik, W., and Slagell, A., "Scrub-PA: A Multi-level, Multi-Dimensional Anonymization Tool for Process Accounting," ACM Computing Research Repository (CoRR), Technical Report cs.CR/0601079, Jan., 2006.
