[LNT] r317833 - docs: Slightly improve custom schema documentation
Matthias Braun via llvm-commits
llvm-commits at lists.llvm.org
Thu Nov 9 12:56:11 PST 2017
Author: matze
Date: Thu Nov 9 12:56:11 2017
New Revision: 317833
URL: http://llvm.org/viewvc/llvm-project?rev=317833&view=rev
Log:
docs: Slightly improve custom schema documentation
- Call the schema 'my_suite' instead of 'size' to avoid confusion with
metrics called 'size'.
- Call the file my_suite.yaml to match the name of the schema.
- Add a sentence about predefined machine/run fields.
Added:
lnt/trunk/docs/my_suite.yaml
- copied, changed from r317746, lnt/trunk/docs/schema-example.yaml
Removed:
lnt/trunk/docs/schema-example.yaml
Modified:
lnt/trunk/docs/importing_data.rst
lnt/trunk/tests/server/db/Inputs/schema-example-migratable.yaml
lnt/trunk/tests/server/db/Inputs/schema-example-nomigration0.yaml
lnt/trunk/tests/server/db/Inputs/schema-example-nomigration1.yaml
lnt/trunk/tests/server/db/Inputs/schema-example-nomigration2.yaml
lnt/trunk/tests/server/db/yamlschema.shtest
Modified: lnt/trunk/docs/importing_data.rst
URL: http://llvm.org/viewvc/llvm-project/lnt/trunk/docs/importing_data.rst?rev=317833&r1=317832&r2=317833&view=diff
==============================================================================
--- lnt/trunk/docs/importing_data.rst (original)
+++ lnt/trunk/docs/importing_data.rst Thu Nov 9 12:56:11 2017
@@ -100,15 +100,14 @@ The `run` information is expected to con
Custom Test Suites
------------------
-LNTs test suites are derived from a set of metadata definitions for each suite.
-Simply put, suites are a collections of metrics that are collected for each run.
-You can define your own test-suites if the schema in a different suite does not
-already meet your needs.
+LNT test suite schemas define which metrics can be tracked for a test and what
+extra information is known about runs and machines. You can define your own
+test suite schemas in a yaml file. The LNT administrator has to place (or
+symlink) this yaml file into the servers schema directory.
-To create a schema place a yaml file into the schemas directory of your lnt
-instance. Example:
+Example:
-.. literalinclude:: schema-example.yaml
+.. literalinclude:: my_suite.yaml
:language: yaml
* LNT currently supports the following metric types:
@@ -120,3 +119,6 @@ instance. Example:
* You need to mark at least 1 of the run fields as ``order: true`` so LNT knows
how to sort runs.
+* Note that runs are not be limited to the fields defined in the schema for
+ the run and machine information. The fields in the schema merely declare which
+ keys get their own column in the database and a prefered treatment in the UI.
Copied: lnt/trunk/docs/my_suite.yaml (from r317746, lnt/trunk/docs/schema-example.yaml)
URL: http://llvm.org/viewvc/llvm-project/lnt/trunk/docs/my_suite.yaml?p2=lnt/trunk/docs/my_suite.yaml&p1=lnt/trunk/docs/schema-example.yaml&r1=317746&r2=317833&rev=317833&view=diff
==============================================================================
--- lnt/trunk/docs/schema-example.yaml (original)
+++ lnt/trunk/docs/my_suite.yaml Thu Nov 9 12:56:11 2017
@@ -1,5 +1,5 @@
format_version: '2'
-name: size
+name: my_suite
metrics:
- name: text_size
bigger_is_better: false
Removed: lnt/trunk/docs/schema-example.yaml
URL: http://llvm.org/viewvc/llvm-project/lnt/trunk/docs/schema-example.yaml?rev=317832&view=auto
==============================================================================
--- lnt/trunk/docs/schema-example.yaml (original)
+++ lnt/trunk/docs/schema-example.yaml (removed)
@@ -1,20 +0,0 @@
-format_version: '2'
-name: size
-metrics:
- - name: text_size
- bigger_is_better: false
- type: Real
- - name: data_size
- bigger_is_better: false
- type: Real
- - name: score
- bigger_is_better: true
- type: Real
- - name: hash
- type: Hash
-run_fields:
- - name: llvm_project_revision
- order: true
-machine_fields:
- - name: hardware
- - name: os
Modified: lnt/trunk/tests/server/db/Inputs/schema-example-migratable.yaml
URL: http://llvm.org/viewvc/llvm-project/lnt/trunk/tests/server/db/Inputs/schema-example-migratable.yaml?rev=317833&r1=317832&r2=317833&view=diff
==============================================================================
--- lnt/trunk/tests/server/db/Inputs/schema-example-migratable.yaml (original)
+++ lnt/trunk/tests/server/db/Inputs/schema-example-migratable.yaml Thu Nov 9 12:56:11 2017
@@ -2,7 +2,7 @@
# over the default example schema. Migration should succeed as this only
# requires the create of new columns.
format_version: "2"
-name: size
+name: my_suite
metrics:
- name: text_size
bigger_is_better: false
Modified: lnt/trunk/tests/server/db/Inputs/schema-example-nomigration0.yaml
URL: http://llvm.org/viewvc/llvm-project/lnt/trunk/tests/server/db/Inputs/schema-example-nomigration0.yaml?rev=317833&r1=317832&r2=317833&view=diff
==============================================================================
--- lnt/trunk/tests/server/db/Inputs/schema-example-nomigration0.yaml (original)
+++ lnt/trunk/tests/server/db/Inputs/schema-example-nomigration0.yaml Thu Nov 9 12:56:11 2017
@@ -1,7 +1,7 @@
# We cannot automatically migrate this as the metric "data_size" metric
# disappears.
format_version: "2"
-name: size
+name: my_suite
metrics:
- name: text_size
bigger_is_better: false
Modified: lnt/trunk/tests/server/db/Inputs/schema-example-nomigration1.yaml
URL: http://llvm.org/viewvc/llvm-project/lnt/trunk/tests/server/db/Inputs/schema-example-nomigration1.yaml?rev=317833&r1=317832&r2=317833&view=diff
==============================================================================
--- lnt/trunk/tests/server/db/Inputs/schema-example-nomigration1.yaml (original)
+++ lnt/trunk/tests/server/db/Inputs/schema-example-nomigration1.yaml Thu Nov 9 12:56:11 2017
@@ -1,7 +1,7 @@
# We cannot automatically migrate this it removes the existing machine field
# "os"
format_version: "2"
-name: size
+name: my_suite
metrics:
- name: text_size
bigger_is_better: false
Modified: lnt/trunk/tests/server/db/Inputs/schema-example-nomigration2.yaml
URL: http://llvm.org/viewvc/llvm-project/lnt/trunk/tests/server/db/Inputs/schema-example-nomigration2.yaml?rev=317833&r1=317832&r2=317833&view=diff
==============================================================================
--- lnt/trunk/tests/server/db/Inputs/schema-example-nomigration2.yaml (original)
+++ lnt/trunk/tests/server/db/Inputs/schema-example-nomigration2.yaml Thu Nov 9 12:56:11 2017
@@ -1,7 +1,7 @@
# We cannot automatically migrate this it removes the existing machine field
# "os"
format_version: "2"
-name: size
+name: my_suite
metrics:
- name: text_size
bigger_is_better: false
Modified: lnt/trunk/tests/server/db/yamlschema.shtest
URL: http://llvm.org/viewvc/llvm-project/lnt/trunk/tests/server/db/yamlschema.shtest?rev=317833&r1=317832&r2=317833&view=diff
==============================================================================
--- lnt/trunk/tests/server/db/yamlschema.shtest (original)
+++ lnt/trunk/tests/server/db/yamlschema.shtest Thu Nov 9 12:56:11 2017
@@ -1,7 +1,7 @@
# RUN: rm -rf "%t.install"
# RUN: lnt create "%t.install"
-# RUN: ln -sf %{src_root}/docs/schema-example.yaml "%t.install/schemas/size.yaml"
-# RUN: lnt import "%t.install" -s size %S/Inputs/customschema-report.json | FileCheck %s
+# RUN: ln -sf %{src_root}/docs/my_suite.yaml "%t.install/schemas/"
+# RUN: lnt import "%t.install" -s my_suite %S/Inputs/customschema-report.json | FileCheck %s
# CHECK: Import succeeded.
# CHECK: Imported Data
@@ -19,34 +19,34 @@
# ===============
#
# Inserting with an extra field shouldn't work just yet
-# RUN: not lnt import "%t.install" -s size "%S/Inputs/customschema-report2.json" 2>&1 | FileCheck %s --check-prefix=NOTUPGRADED
+# RUN: not lnt import "%t.install" -s my_suite "%S/Inputs/customschema-report2.json" 2>&1 | FileCheck %s --check-prefix=NOTUPGRADED
# NOTUPGRADED: Metric u'newfield' unknown in suite
# Upgrading to a schema with metrics/fields removed should fail
-# RUN: rm -f "%t.install/schemas/size.yaml"
-# RUN: ln -sf "%S/Inputs/schema-example-nomigration0.yaml" "%t.install/schemas/size.yaml"
-# RUN: not lnt import "%t.install" -s size "%S/Inputs/customschema-report.json" 2>&1 | FileCheck %s --check-prefix=NOMIGRATION0
+# RUN: rm -f "%t.install/schemas/my_suite.yaml"
+# RUN: ln -sf "%S/Inputs/schema-example-nomigration0.yaml" "%t.install/schemas/my_suite.yaml"
+# RUN: not lnt import "%t.install" -s my_suite "%S/Inputs/customschema-report.json" 2>&1 | FileCheck %s --check-prefix=NOMIGRATION0
# NOMIGRATION0: Cannot automatically migrate database: Metrics removed: data_size
#
-# RUN: rm -f "%t.install/schemas/size.yaml"
-# RUN: ln -sf "%S/Inputs/schema-example-nomigration1.yaml" "%t.install/schemas/size.yaml"
-# RUN: not lnt import "%t.install" -s size "%S/Inputs/customschema-report.json" 2>&1 | FileCheck %s --check-prefix=NOMIGRATION1
+# RUN: rm -f "%t.install/schemas/my_suite.yaml"
+# RUN: ln -sf "%S/Inputs/schema-example-nomigration1.yaml" "%t.install/schemas/my_suite.yaml"
+# RUN: not lnt import "%t.install" -s my_suite "%S/Inputs/customschema-report.json" 2>&1 | FileCheck %s --check-prefix=NOMIGRATION1
# NOMIGRATION1: Cannot automatically migrate database: Machine fields removed: os
#
-# RUN: rm -f "%t.install/schemas/size.yaml"
-# RUN: ln -sf "%S/Inputs/schema-example-nomigration2.yaml" "%t.install/schemas/size.yaml"
-# RUN: not lnt import "%t.install" -s size "%S/Inputs/customschema-report.json" 2>&1 | FileCheck %s --check-prefix=NOMIGRATION2
+# RUN: rm -f "%t.install/schemas/my_suite.yaml"
+# RUN: ln -sf "%S/Inputs/schema-example-nomigration2.yaml" "%t.install/schemas/my_suite.yaml"
+# RUN: not lnt import "%t.install" -s my_suite "%S/Inputs/customschema-report.json" 2>&1 | FileCheck %s --check-prefix=NOMIGRATION2
# NOMIGRATION2: Cannot automatically migrate database: Type mismatch in metric 'data_size'
# This upgrade should finally work
-# RUN: rm -f "%t.install/schemas/size.yaml"
-# RUN: ln -sf "%S/Inputs/schema-example-migratable.yaml" "%t.install/schemas/size.yaml"
-# RUN: lnt import "%t.install" "%S/Inputs/customschema-report2.json" -s size --show-sql 2>&1 | FileCheck %s --check-prefix=MIGRATION
-#
-# MIGRATION: ALTER TABLE "size_Sample" ADD COLUMN newfield FLOAT
-# MIGRATION: ALTER TABLE "size_Run" ADD COLUMN new_run_field VARCHAR(256)
-# MIGRATION: ALTER TABLE "size_Machine" ADD COLUMN new_machine_field VARCHAR(256)
+# RUN: rm -f "%t.install/schemas/my_suite.yaml"
+# RUN: ln -sf "%S/Inputs/schema-example-migratable.yaml" "%t.install/schemas/my_suite.yaml"
+# RUN: lnt import "%t.install" "%S/Inputs/customschema-report2.json" -s my_suite --show-sql 2>&1 | FileCheck %s --check-prefix=MIGRATION
+#
+# MIGRATION: ALTER TABLE "my_suite_Sample" ADD COLUMN newfield FLOAT
+# MIGRATION: ALTER TABLE "my_suite_Run" ADD COLUMN new_run_field VARCHAR(256)
+# MIGRATION: ALTER TABLE "my_suite_Machine" ADD COLUMN new_machine_field VARCHAR(256)
#
# MIGRATION: Import succeeded.
# MIGRATION: Imported Data
More information about the llvm-commits
mailing list