I was running ATS_File_Op test on my storage array. Approximate time to complete Create-Delete test was 66 seconds. Time difference between with and without Hardware Acceleration was always around +/-100ms. 1 out of 2 runs would fail certification. I spooked through vmware test scripts and found test would sleep for 60 seconds between polling for command completion status.
I rerun the test after modifying the line in below code snipped from $wait_time *= 5; to $wait_time = 1; This change would make polling interval to be 1 second.
Now time difference between with and without Hardware Acceleration is always 4 seconds. Every time my storage box passed certification criteria. Approx time to complete Create-Delete test is 30 seconds. However I can't submit for certification as I modified vmware signed file.
Conclusion: If time difference between with and without Hardware Acceleration is less than 60 seconds, then vmware test suite couldnot pickup accurate completion time leading to incorrectly pass or fail the test.
If my findings are correct, then vmware certification suite will incorrectly pass or fail vmware ready certification.
I am happy to be proved wrong.
Code snippet for reference:
sub RunFileCreateDeleteTest
{
----
---
if (VTAF::Test::Storage::StorageCert::FVT::VAAI::Utils::RunCmdListOnHosts($cmdref, $cmdref, 3600, $testparams))
---
---
}
sub RunCmdListOnHosts
{
---
---
while ($wait < $total_wait)
{
# let us check commands on Host1
foreach $cmditem ( @host1_Cmd )
{
if( $cmditem->GetState() eq CMDSTATE->{COMPLETED} )
{
---
---
# Let us check commands on Host2
foreach $cmditem ( @host2_Cmd )
{
if( $cmditem->GetState() eq CMDSTATE->{COMPLETED}
---
---
}
if ( 0 == scalar(@host1_Cmd) && 0 == scalar(@host2_Cmd) )
{
$testcomplete = 1;
last;
}
my $wait_time = int(($total_wait-$wait)/300) ;
if ($wait_time == 0)
{
$wait_time = 1;
}
$wait_time *= 5;
sleep $wait_time;
$wait += $wait_time;
---
---
}