[PATCH] Make LNT compatible with PostgreSQL
Yi Kong
kongy.dev at gmail.com
Thu Jul 31 04:34:06 PDT 2014
I've updated the patch to handle failures, and expanded tests.
-Yi
On 28 July 2014 16:15, Yi Kong <kongy.dev at gmail.com> wrote:
> Hi Chris,
>
> Sorry for the delay, I've been occupied by some other work.
>
> Can you try this test and see if it works on your Jenkins? I will
> expand the test coverage if it proves to be working.
>
> Cheers,
>
> Yi
>
> On 24 July 2014 00:29, Chris Matthews <chris.matthews at apple.com> wrote:
>> The lit test suite supports a requires clause. This allows you to do this
>>
>> # REQUIRES: postgres
>>
>> In a test, then that test is only run if a —param postgres=True is passed to the lit invocation. This allows us to keep the tests disabled for all except people who want to run the extra postgres tests.
>>
>> I have a Jenkins bot, which I could install postgres on, so we would have coverage on postgres there.
>>
>> Would it be possible to start the lit script off with a psql command which drop/creates a database first? so the database name and URL are known ahead of time.
>>
>>> On Jul 23, 2014, at 7:41 AM, Yi Kong <kongy.dev at gmail.com> wrote:
>>>
>>> On 21 July 2014 23:49, Chris Matthews <chris.matthews at apple.com> wrote:
>>>> Okay. That is fine. I think it is okay to tell people to start fresh if they are going to switch.
>>>>
>>>> Go ahead and commit your patch to add the Postgres creation capability.
>>>>
>>>> Can you add a little blurb to docs/tools.rst explaining how to create for Postgres.
>>>>
>>>> Could you also add a Postgres requires directive to the LNT test suite (off by default),
>>>
>>> I'm not sure what do you mean. Are you suggesting adding 'psycopg2' to
>>> install_requires?
>>>
>>>> and make a new test in lnttool to test this code path?
>>>
>>> It's not easily testable as Postgres doesn't support in-memory scratch database.
>>>
>>> Thanks for reviewing!
>>>
>>> -Yi
>>
-------------- next part --------------
Index: docs/intro.rst
===================================================================
--- docs/intro.rst (revision 214398)
+++ docs/intro.rst (working copy)
@@ -34,13 +34,13 @@
If you want to run an LNT server, you will need to perform the following
additional steps:
- 2. Create a new LNT installation:
+ 2. Create a new LNT installation::
lnt create path/to/install-dir
This will create the LNT configuration file, the default database, and a
.wsgi wrapper to create the application. You can execute the generated app
- directly to run with the builtin web server, or use
+ directly to run with the builtin web server, or use::
lnt runserver path/to/install-dir
@@ -62,7 +62,22 @@
If running in a virtualenv you will need to configure that as well; see the
`modwsgi wiki <http://code.google.com/p/modwsgi/wiki/VirtualEnvironments>`_.
+For production servers, you should consider using a full DBMS like PostgreSQL.
+To create an LNT instance with PostgreSQL backend, you need to do this instead:
+ 1. Create an LNT database in PostgreSQL, also make sure the user has
+ write permission to the database::
+
+ CREATE DATABASE "lnt.db"
+
+ 2. Then create LNT installation::
+
+ lnt create path/to/install-dir --db-dir postgresql://user@host
+
+ 3. Run server normally::
+
+ lnt runserver path/to/install-dir
+
Development
-----------
@@ -86,7 +101,6 @@
if you prefer 'unittest' style output (this still requires that 'lit' be
installed).
-
Architecture
------------
Index: docs/tools.rst
===================================================================
--- docs/tools.rst (revision 214398)
+++ docs/tools.rst (working copy)
@@ -51,7 +51,8 @@
to tweak the generated server, but they can all be modified after the fact
in the LNT configuration file.
- The default server will have one database named *default*.
+ The default server will have a sqlite3 database named *default*. You can
+ specify to use PostgreSQL using ``--db-dir postgresql://user@hostname``.
``lnt createdb <path>``
Creates a new LNT sqlite3 database at the specified path.
Index: lnt/lnttool/create.py
===================================================================
--- lnt/lnttool/create.py (revision 214398)
+++ lnt/lnttool/create.py (working copy)
@@ -151,9 +151,7 @@
hosturl = "http://%s/%s" % (hostname, hostsuffix)
python_executable = sys.executable
- db_dir_path = os.path.join(basepath, db_dir)
cfg_path = os.path.join(basepath, config)
- db_path = os.path.join(db_dir_path, default_db)
tmp_path = os.path.join(basepath, tmp_dir)
wsgi_path = os.path.join(basepath, wsgi)
secret_key = (opts.secret_key or
@@ -160,9 +158,16 @@
hashlib.sha1(str(random.getrandbits(256))).hexdigest())
os.mkdir(path)
- os.mkdir(db_dir_path)
os.mkdir(tmp_path)
+ # If the path does not contain database type, assume relative path.
+ if "://" not in db_dir:
+ db_dir_path = os.path.join(basepath, db_dir)
+ db_path = os.path.join(db_dir_path, default_db)
+ os.mkdir(db_dir_path)
+ else:
+ db_path = os.path.join(db_dir, default_db)
+
cfg_version = kConfigVersion
cfg_file = open(cfg_path, 'w')
cfg_file.write(kConfigTemplate % locals())
Index: lnt/server/config.py
===================================================================
--- lnt/server/config.py (revision 214398)
+++ lnt/server/config.py (working copy)
@@ -90,8 +90,10 @@
default_email_config = EmailConfig(False, '', '', [])
dbDir = data.get('db_dir', '.')
- dbDirPath = os.path.join(baseDir, dbDir)
+ # If the path does not contain database type, assume relative path.
+ dbDirPath = dbDir if "://" in dbDir else os.path.join(baseDir, dbDir)
+
# FIXME: Remove this default.
tempDir = data.get('tmp_dir', 'viewer/resources/graphs')
Index: tests/lit.cfg
===================================================================
--- tests/lit.cfg (revision 214398)
+++ tests/lit.cfg (working copy)
@@ -38,6 +38,9 @@
if lit_config.params.get('long', None):
config.available_features.add('long')
+if lit_config.params.get('postgres', None):
+ config.available_features.add('postgres')
+
# Enable coverage.py reporting, assuming the coverage module has been installed
# and sitecustomize.py in the virtualenv has been modified appropriately.
if lit_config.params.get('check-coverage', None):
Index: tests/lnttool/PostgresDB.py
===================================================================
--- tests/lnttool/PostgresDB.py (revision 0)
+++ tests/lnttool/PostgresDB.py (working copy)
@@ -0,0 +1,42 @@
+# RUN: rm -rf %t.install
+# RUN: dropdb --if-exists testdb
+# RUN: createdb testdb
+
+# RUN: lnt create %t.install --db-dir postgresql://localhost/ --default-db testdb
+
+# Import a test set.
+# RUN: lnt import %t.install %{shared_inputs}/sample-a-small.plist \
+# RUN: --commit=1 --show-sample-count
+
+# Import a test set.
+# RUN: lnt import %t.install %{shared_inputs}/sample-a-small.plist \
+# RUN: --commit=1 --show-sample-count
+
+# Check that we remove both the sample and the run, and that we don't commit by
+# default.
+#
+# RUN: lnt updatedb %t.install --testsuite nts \
+# RUN: --delete-run 1 --show-sql > %t.out
+# RUN: FileCheck --check-prefix CHECK-RUNRM %s < %t.out
+
+# CHECK-RUNRM: DELETE FROM "NT_Sample" WHERE "NT_Sample"."RunID" IN (%(RunID_1)s)
+# CHECK-RUNRM-NEXT: {'RunID_1': 1}
+# CHECK-RUNRM: DELETE FROM "NT_Run" WHERE "NT_Run"."ID" IN (%(ID_1)s)
+# CHECK-RUNRM-NEXT: {'ID_1': 1}
+# CHECK-RUNRM: ROLLBACK
+
+# Check that we remove runs when we remove a machine.
+#
+# RUN: lnt updatedb %t.install --testsuite nts \
+# RUN: --delete-machine "LNT SAMPLE MACHINE" --commit=1 --show-sql > %t.out
+# RUN: FileCheck --check-prefix CHECK-MACHINERM %s < %t.out
+
+# CHECK-MACHINERM: DELETE FROM "NT_Sample" WHERE "NT_Sample"."RunID" IN (%(RunID_1)s)
+# CHECK-MACHINERM-NEXT: {'RunID_1': 1}
+# CHECK-MACHINERM: DELETE FROM "NT_Run" WHERE "NT_Run"."ID" IN (%(ID_1)s)
+# CHECK-MACHINERM-NEXT: {'ID_1': 1}
+# CHECK-MACHINERM: DELETE FROM "NT_Machine" WHERE "NT_Machine"."Name" = %(Name_1)s
+# CHECK-MACHINERM-NEXT: {'Name_1': 'LNT SAMPLE MACHINE'}
+# CHECK-MACHINERM: COMMIT
+
+# REQUIRES: postgres
More information about the llvm-commits
mailing list