[LLVMdev] LNT compile-time performance testing

[Michael Gottesman]

There is currently no one publicly using the compile tests. We use it internally around the clock. I am not sure what Daniel's vision for the use of this externally (i.e. I am not sure how beta/non-beta he thinks this is), but regardless the way to use it is:

1. Create a directory.
2. Stick a bunch of tar balls you want into the directory.
3. Create a project_list.json file and write a configuration of the form:
{
	"single-file" : {
		"tests" : [
			{
				"path" : "path to single source file",
				"pch_input" : "PCH if there is one"
				"extra_flags": [
					"flag1", "flag2", ...
				]
			},
			{
				...
			}
		],
		"pch" : [
			{
				"path" : "PCH Header",
				"name" : "PCH Name",
				"output": "PCH Output name"
			}
		]
	},
	"projects" : [
		{
			"name" : "NAME",
			"archive": "ARCHIVE",   **I believe only zip/.tar.gz are supported now,
			"patch_files" : [ "Patch 1 to apply to project", "Patch 2 to apply to project"],
			"build_info" : {
				"style" : "make",
				"target" : "MAKE RULE" (i.e. all/etc),
				"file": "PATH TO UNZIPPED DIRECTORY" **(i.e. it assumes you tar balled up a whole project directory so if X.tar.gz is the archive then X is the directory [this name is not the best but I might be remembering things incorrectly]),
				"config" : {
					"Release" : {
						"extra_env" : {
							"EXTRA_ENV_VAR" : "EXTRA_ENV_VALUE",
							...
						},
						"extra_args" : [

						],
					},
					"Debug" : {
						SAME AS RELEASE
					}
				}
			}
		},
		{
			NEXT_PROJECT ...
		}
	]
}

Note that currently it only does make and Xcode-project builds (I only showed the interesting args for the makefiles since I am assuming you are not interested in Xcode style builds.

On Dec 19, 2012, at 2:05 PM, Eli Bendersky <eliben at google.com> wrote:

> Hi,
> 
> I was looking at LNT's "compile" tests, listed here:
> 
> $ lnt-sandbox/bin/lnt showtests
> Available tests:
>  compile - Single file compile-time performance testing
>  nt      - LLVM test-suite compile and execution tests
> 
> When trying to run it, I get Python exceptions in the code trying to
> parse the output of 'ifconfig' in order to infer my MAC address (this
> is on 64-bit Ubuntu 12.04). That parsing code indeed looks strange.
> 
> Is the "compile" test in a working condition at all? How should I run
> it? (I don't think the LNT docs say much about it, correct me if I'm
> wrong). Thanks in advance, Eli.