<html>

    <head>

      <base href="https://bugs.llvm.org/">

    </head>

    <body><table border="1" cellspacing="0" cellpadding="8">

        <tr>

          <th>Bug ID</th>

          <td><a class="bz_bug_link 

          bz_status_NEW "

   title="NEW - [OpenMP] Wrong answers with nested 'omp parallel for'"

   href="https://bugs.llvm.org/show_bug.cgi?id=52169">52169</a>

          </td>

        </tr>

        <tr>

          <th>Summary</th>

          <td>[OpenMP] Wrong answers with nested 'omp parallel for'

          </td>

        </tr>

        <tr>

          <th>Product</th>

          <td>OpenMP

          </td>

        </tr>

        <tr>

          <th>Version</th>

          <td>unspecified

          </td>

        </tr>

        <tr>

          <th>Hardware</th>

          <td>PC

          </td>

        </tr>

        <tr>

          <th>OS</th>

          <td>Linux

          </td>

        </tr>

        <tr>

          <th>Status</th>

          <td>NEW

          </td>

        </tr>

        <tr>

          <th>Severity</th>

          <td>normal

          </td>

        </tr>

        <tr>

          <th>Priority</th>

          <td>P

          </td>

        </tr>

        <tr>

          <th>Component</th>

          <td>Clang Compiler Support

          </td>

        </tr>

        <tr>

          <th>Assignee</th>

          <td>unassignedclangbugs@nondot.org

          </td>

        </tr>

        <tr>

          <th>Reporter</th>

          <td>Dan.Palermo@amd.com

          </td>

        </tr>

        <tr>

          <th>CC</th>

          <td>llvm-bugs@lists.llvm.org

          </td>

        </tr></table>

      <p>

        <div>

        <pre>Created <span class=""><a href="attachment.cgi?id=25361" name="attach_25361" title="SNAP reduced test case">attachment 25361</a> <a href="attachment.cgi?id=25361&action=edit" title="SNAP reduced test case">[details]</a></span>

SNAP reduced test case

Wrong answers with nested 'omp parallel for'

Attached are two self checking tests that demonstrate a wrong answer that

occurs with nested 'omp parallel for'. 

1) Original reduced test case from the SNAP application:

   snap_red.c

This produces wrong answers when there is a nested 'omp parallel for' pragma.

The expected behavior is that the inner 'parallel for' should serialize.

2) Since the SNAP reduced test case still contains reduction, atomic, collapse

as well as thread limits, a separate test that just uses only 'omp parallel

for'

was constructed: initialize a 3D array (to keep a similar loop structure):

   nested_par.c

This also fails in a similar manner, but examining 2-D slices of the array

shows that the inner serialized loop is not executing all iterations..

Below are observations made using these two tests.

snap_red.c (reduction)

-----------------------------------------------------------------------------

Correct answers are produced if the inner 'omp parallel for' is commented out:

$ clang -O2 -fopenmp -fopenmp-targets=nvptx64 -Xopenmp-target=nvptx64

-march=sm_70 snap_red.c -o snap_red -L/usr/local/cuda/targets/x86_64-linux/lib

-lcudart

$ ./snap_red

The result is = 35100000 exp:35100000!

-----------------------------------------------------------------------------

Wrong answers seen when inner 'omp parallel for' is present:

$ clang -O2 -fopenmp -fopenmp-targets=nvptx64 -Xopenmp-target=nvptx64

-march=sm_70 snap_red.c -o snap_red -L/usr/local/cuda/targets/x86_64-linux/lib

-lcudart

$ ./snap_red

The result is = 225000 exp:35100000!

Failed 225000

-----------------------------------------------------------------------------

Abort seen when using the new runtime:

$ clang -O2 -fopenmp -fopenmp-targets=nvptx64 -Xopenmp-target=nvptx64

-march=sm_70 snap_red.c -o snap_red -L/usr/local/cuda/targets/x86_64-linux/lib

-lcudart -fopenmp-target-new-runtime

$ ./snap_red

CUDA error: an illegal memory access was encountered

Libomptarget error: Copying data from device failed.

Libomptarget error: Call to targetDataEnd failed, abort target.

Libomptarget error: Failed to process data after launching the kernel.

Libomptarget error: Run with LIBOMPTARGET_INFO=4 to dump host-target pointer

mappings.

Libomptarget error: Source location information not present. Compile with -g or

-gline-tables-only.

Libomptarget fatal error 1: failure of target construct while offloading is

mandatory

Aborted (core dumped)

-----------------------------------------------------------------------------

Different wrong answer seen with 'if (0)' added to inner 'omp parallel for'

note: this is also the same behavior seen if -O2 is not used

$ clang -O2 -fopenmp -fopenmp-targets=nvptx64 -Xopenmp-target=nvptx64

-march=sm_70 snap_red.c -o snap_red -L/usr/local/cuda/targets/x86_64-linux/lib

-lcudart

$ ./snap_red

The result is = 1650000 exp:35100000!

Failed 1650000

-----------------------------------------------------------------------------

Wrong answer seen with smaller values

  ng   = 5;

  cmom = 10;

  nxyz = 20;

$ clang -O2 -fopenmp -fopenmp-targets=nvptx64 -Xopenmp-target=nvptx64

-march=sm_70 snap_red.c -o snap_red -L/usr/local/cuda/targets/x86_64-linux/lib

-lcudart

$ ./snap_red

The result is = 900 exp:40500!

Failed 900

-----------------------------------------------------------------------------

nested_par.c (3-D array init)

-----------------------------------------------------------------------------