Benchmarks

Our goal is to assemble a benchmark suite that will be capable of measuring changes made to a system. We would like to write a technical report describing why and how to use the benchmark suite.

• Patrick Bridges
• Nathan Graham
• Philip Soltero

Nathan is having problems running the fortran benchmarks. They are all failing when he runs them.
—Philip Soltero 04/24/07 23:03

Nathan and I have encountered this problem:

[ngraham@phoenix bin]$ mpirun -np 16 --hostfile MyHostFile --mca btl
self,sm,openib bt.B.16.mpi_io_full


NAS Parallel Benchmarks 3.2 -- BT Benchmark

No input file inputbt.data. Using compiled defaults
Size: 102x102x102
Iterations: 200    dt:   0.000300
Number of active processes:    16

BTIO -- FULL MPI-IO write interval:  5

ADIOI_NFS_OPEN (line 55): **filenoexist btio.full.outADIOI_NFS_OPEN (line
55): **filenoexist btio.full.outADIOI_NFS_OPEN (line 55): **filenoexist
btio.full.outADIO_OPEN (line 273): **oremote_fail
Error opening file
Error opening file
ADIO_OPEN (line 273): **oremote_fail
ADIO_OPEN (line 273): **oremote_fail
ADIO_OPEN (line 273): **oremote_fail
ADIO_OPEN (line 273): **oremote_fail
ADIO_OPEN (line 273): **oremote_fail
Error opening file
ADIOI_NFS_OPEN (line 55): **filenoexist btio.full.outADIOI_NFS_OPEN (line
55): **filenoexist btio.full.outMPI_FILE_CLOSE (line 51): **iobadfh
ADIOI_NFS_OPEN (line 55): **filenoexist btio.full.outADIOI_NFS_OPEN (line
55): **filenoexist btio.full.outMPI_FILE_CLOSE (line 51): **iobadfh
Error opening file
Error opening file
Error opening file
Error opening file
Error opening file
ADIO_OPEN (line 273): **oremote_fail
MPI_FILE_CLOSE (line 51): **iobadfh
Error opening file
Error opening file
MPI_FILE_CLOSE (line 51): **iobadfh
MPI_FILE_CLOSE (line 51): **iobadfh
MPI_FILE_CLOSE (line 51): **iobadfh
MPI_FILE_CLOSE (line 51): **iobadfh
MPI_FILE_CLOSE (line 51): **iobadfh
Error opening file
ADIOI_NFS_OPEN (line 55): **filenoexist btio.full.outMPI_FILE_CLOSE (line
51): **iobadfh
MPI_FILE_CLOSE (line 51): **iobadfh
Error opening file
Error opening file
Error opening file
Error opening file
Error opening file
MPI_FILE_CLOSE (line 51): **iobadfh
ADIO_OPEN (line 273): **oremote_fail
MPI_FILE_CLOSE (line 51): **iobadfh
ADIO_OPEN (line 273): **oremote_fail
MPI_FILE_CLOSE (line 51): **iobadfh
ADIO_OPEN (line 273): **oremote_fail
MPI_FILE_CLOSE (line 51): **iobadfh
ADIO_OPEN (line 273): **oremote_fail
MPI_FILE_CLOSE (line 51): **iobadfh
ADIO_OPEN (line 273): **oremote_fail
MPI_FILE_CLOSE (line 51): **iobadfh

It occurs sporadically, and might be due to NFS not being able to inform all nodes that a file has been created before a node opens the file. I've tried unsuccessfully to coerce NFS into updating each node (by performing an ls before opening the file), but that did not work. I don't know enough about NFS to solve the problem, so I might turn to looking at what the BT benchmark is doing when it initially opens the file.
—Philip Soltero 04/16/07 23:19

Past Problems

This is a preliminary outline of tasks that need to be accomplished in order to achieve our goal.

• Run benchmarks on Phoenix and collect data.
• After collecting a sufficient amount of data, upgrade the InfiniBand stack.
  • Would be wise to make a system image of Phoenix before and after upgrading the InfiniBand stack.
• Re-run all benchmarks to determine the affect of upgrading the InfiniBand stack.
• Determine what each benchmark is measuring so that each benchmark can be categorized by what it measures.
• Assemble a benchmark suite from which benchmarks are used to measure a specific metric.
• Make changes to the system and run benchmarks from the suite that are capable of measuring the change.
• Write a technical report documenting the relevance and usage of the benchmark suite.