[cfe-dev] SAMATE Juliet Test Suite for clang and clang analyzer

[Edoardo P.]

Hello! I'd like to suggest to use the Juliet Test Suite for C/C++, v1.2
from NIST site: http://samate.nist.gov/SRD/testsuite.php (last table). It's
at least 100 MB as a zip file, at least 600 MB when unzipped.

It contains various files:

- compile.bat file to compile with windows
- Makefile_all for non-Windows (I tried to run but managed to get full
memory and locked my system for a certain time, no idea why)
- some python scripts to regenerate the builders

three folders:
- doc, which has a pdf manual with a lot of info
- testcasesupport, with headers and sources which are common to all tests
and it should be included when compiling the tests
- testcases, with all the tests


This last folder, in turn, is divided in folders per CWE types.

My first question is: which ones are you interested to investigate? The
list is attached as dir_list.txt.

Be aware that the cwe dirs attached as dir_list_w32.txt contain only win32
tests.

This way, I can generate a file_list.txt which will contain only the
selected test cases and use it to run clang (+ static analyzer) and check
for false positives/false negatives.

Edward-san
-- 
Mathematics is the language with which God has written the universe.
(Galilei)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20140202/12dc0da0/attachment.html>
-------------- next part --------------
CWE15_External_Control_of_System_or_Configuration_Setting
CWE23_Relative_Path_Traversal
CWE36_Absolute_Path_Traversal
CWE78_OS_Command_Injection
CWE90_LDAP_Injection
CWE114_Process_Control
CWE121_Stack_Based_Buffer_Overflow
CWE122_Heap_Based_Buffer_Overflow
CWE123_Write_What_Where_Condition
CWE124_Buffer_Underwrite
CWE126_Buffer_Overread
CWE127_Buffer_Underread
CWE134_Uncontrolled_Format_String
CWE176_Improper_Handling_of_Unicode_Encoding
CWE188_Reliance_on_Data_Memory_Layout
CWE190_Integer_Overflow
CWE191_Integer_Underflow
CWE194_Unexpected_Sign_Extension
CWE195_Signed_to_Unsigned_Conversion_Error
CWE196_Unsigned_to_Signed_Conversion_Error
CWE197_Numeric_Truncation_Error
CWE222_Truncation_of_Security_Relevant_Information
CWE223_Omission_of_Security_Relevant_Information
CWE226_Sensitive_Information_Uncleared_Before_Release
CWE242_Use_of_Inherently_Dangerous_Function
CWE244_Heap_Inspection
CWE247_Reliance_on_DNS_Lookups_in_Security_Decision
CWE252_Unchecked_Return_Value
CWE253_Incorrect_Check_of_Function_Return_Value
CWE256_Plaintext_Storage_of_Password
CWE259_Hard_Coded_Password
CWE272_Least_Privilege_Violation
CWE273_Improper_Check_for_Dropped_Privileges
CWE284_Improper_Access_Control
CWE319_Cleartext_Tx_Sensitive_Info
CWE321_Hard_Coded_Cryptographic_Key
CWE325_Missing_Required_Cryptographic_Step
CWE327_Use_Broken_Crypto
CWE328_Reversible_One_Way_Hash
CWE338_Weak_PRNG
CWE364_Signal_Handler_Race_Condition
CWE366_Race_Condition_Within_Thread
CWE367_TOC_TOU
CWE369_Divide_by_Zero
CWE377_Insecure_Temporary_File
CWE390_Error_Without_Action
CWE391_Unchecked_Error_Condition
CWE396_Catch_Generic_Exception
CWE397_Throw_Generic_Exception
CWE398_Poor_Code_Quality
CWE400_Resource_Exhaustion
CWE401_Memory_Leak
CWE404_Improper_Resource_Shutdown
CWE415_Double_Free
CWE416_Use_After_Free
CWE426_Untrusted_Search_Path
CWE427_Uncontrolled_Search_Path_Element
CWE440_Expected_Behavior_Violation
CWE457_Use_of_Uninitialized_Variable
CWE459_Incomplete_Cleanup
CWE464_Addition_of_Data_Structure_Sentinel
CWE467_Use_of_sizeof_on_Pointer_Type
CWE468_Incorrect_Pointer_Scaling
CWE469_Use_of_Pointer_Subtraction_to_Determine_Size
CWE475_Undefined_Behavior_for_Input_to_API
CWE476_NULL_Pointer_Dereference
CWE478_Missing_Default_Case_in_Switch
CWE479_Signal_Handler_Use_of_Non_Reentrant_Function
CWE480_Use_of_Incorrect_Operator
CWE481_Assigning_Instead_of_Comparing
CWE482_Comparing_Instead_of_Assigning
CWE483_Incorrect_Block_Delimitation
CWE484_Omitted_Break_Statement_in_Switch
CWE500_Public_Static_Field_Not_Final
CWE506_Embedded_Malicious_Code
CWE510_Trapdoor
CWE511_Logic_Time_Bomb
CWE526_Info_Exposure_Environment_Variables
CWE534_Info_Exposure_Debug_Log
CWE535_Info_Exposure_Shell_Error
CWE546_Suspicious_Comment
CWE561_Dead_Code
CWE562_Return_of_Stack_Variable_Address
CWE563_Unused_Variable
CWE570_Expression_Always_False
CWE571_Expression_Always_True
CWE587_Assignment_of_Fixed_Address_to_Pointer
CWE588_Attempt_to_Access_Child_of_Non_Structure_Pointer
CWE590_Free_Memory_Not_on_Heap
CWE591_Sensitive_Data_Storage_in_Improperly_Locked_Memory
CWE605_Multiple_Binds_Same_Port
CWE606_Unchecked_Loop_Condition
CWE615_Info_Exposure_by_Comment
CWE617_Reachable_Assertion
CWE620_Unverified_Password_Change
CWE665_Improper_Initialization
CWE666_Operation_on_Resource_in_Wrong_Phase_of_Lifetime
CWE667_Improper_Locking
CWE672_Operation_on_Resource_After_Expiration_or_Release
CWE674_Uncontrolled_Recursion
CWE675_Duplicate_Operations_on_Resource
CWE676_Use_of_Potentially_Dangerous_Function
CWE680_Integer_Overflow_to_Buffer_Overflow
CWE681_Incorrect_Conversion_Between_Numeric_Types
CWE685_Function_Call_With_Incorrect_Number_of_Arguments
CWE688_Function_Call_With_Incorrect_Variable_or_Reference_as_Argument
CWE690_NULL_Deref_From_Return
CWE758_Undefined_Behavior
CWE761_Free_Pointer_Not_at_Start_of_Buffer
CWE762_Mismatched_Memory_Management_Routines
CWE773_Missing_Reference_to_Active_File_Descriptor_or_Handle
CWE775_Missing_Release_of_File_Descriptor_or_Handle
CWE780_Use_of_RSA_Algorithm_Without_OAEP
CWE785_Path_Manipulation_Function_Without_Max_Sized_Buffer
CWE789_Uncontrolled_Mem_Alloc
CWE832_Unlock_of_Resource_That_is_Not_Locked
CWE835_Infinite_Loop
CWE843_Type_Confusion
-------------- next part --------------
CWE15_External_Control_of_System_or_Configuration_Setting
CWE90_LDAP_Injection
CWE114_Process_Control
CWE176_Improper_Handling_of_Unicode_Encoding
CWE222_Truncation_of_Security_Relevant_Information
CWE223_Omission_of_Security_Relevant_Information
CWE226_Sensitive_Information_Uncleared_Before_Release
CWE244_Heap_Inspection
CWE247_Reliance_on_DNS_Lookups_in_Security_Decision
CWE256_Plaintext_Storage_of_Password
CWE259_Hard_Coded_Password
CWE272_Least_Privilege_Violation
CWE273_Improper_Check_for_Dropped_Privileges
CWE284_Improper_Access_Control
CWE319_Cleartext_Tx_Sensitive_Info
CWE321_Hard_Coded_Cryptographic_Key
CWE325_Missing_Required_Cryptographic_Step
CWE327_Use_Broken_Crypto
CWE328_Reversible_One_Way_Hash
CWE338_Weak_PRNG
CWE534_Info_Exposure_Debug_Log
CWE535_Info_Exposure_Shell_Error
CWE591_Sensitive_Data_Storage_in_Improperly_Locked_Memory
CWE615_Info_Exposure_by_Comment
CWE620_Unverified_Password_Change
CWE780_Use_of_RSA_Algorithm_Without_OAEP
CWE785_Path_Manipulation_Function_Without_Max_Sized_Buffer