Run repeated tests reported as different tests

Question

Run repeated tests reported as different tests

Navigation

#1 by (1 votes)

1

I would like to do unit tests of a function with different expected inputs and outputs.

My function is irrelevant, so instead I will use an example function that counts English words with the following cadidata implementation:

int countEnglishWords( const std::string& text )
{
  return 5;
};

The following would be the test data. The end of the test data is marked with an element with the word "END".

struct TestData {
  std::string text;
  int englishWords;
};

struct TestData data[] = // Marcar final con "END"
{
  { "The car is very fast", 5 },
  { "El coche es muy rapido", 0 },
  { "The rain in Spain stays mainly in the plain", 9},
  { "XXXXX OOOOO TTTT", 0},
  { "Yes Si No No", 3},
  { "I have a cheerful live", 5},
  { "END", 0}
};

I could easily write 6 test cases and I would get the result I want. But this is not maintainable, given that future tests added would not be tested, it required new testcases, which would be mainly fill code (boilerplate). So I have written a single test case that runs in a loop all the test data such that:

#include <cppunit/ui/text/TestRunner.h>
#include <cppunit/extensions/HelperMacros.h>

class cppUnit_test: public CppUnit::TestFixture
{
private:
   CPPUNIT_TEST_SUITE (cppUnit_test);
   CPPUNIT_TEST(myTest);
   CPPUNIT_TEST_SUITE_END();

public:
   void myTest();
};

void cppUnit_test::myTest()
{
  TestData* p = data;
  while ( p->text != "END")
  {
    std::stringstream ss;
    ss << "Text=\"" << p->text << "\" Counted=" << 
       countEnglishWords(p->text) << " Expected=" << p->englishWords;
    CPPUNIT_ASSERT_MESSAGE( ss.str().c_str(), 
                countEnglishWords(p->text) == p->englishWords );
    ++p;
  }
}

int main()
{
   CPPUNIT_TEST_SUITE_REGISTRATION (cppUnit_test);
   CppUnit::Test *suite =
         CppUnit::TestFactoryRegistry::getRegistry().makeTest();
   CppUnit::TextUi::TestRunner runner;
   runner.addTest(suite);
   runner.run();
   return 0;
}

The problem is that the previous code executes the first test well and also detects the error in the second test but stops after the second test. The report it generates is:

!!! FAILURES !!!
  Test Results:
  Run: 1 Failures: 1 Errors: 0

But the result I would like is:

!!! FAILURES !!!
  Test Results:
  Run: 6 Failures: 4 Errors: 0

c++ pruebas-unitarias

asked by Jose Antonio Dura Olmos 02.06.2017 в 10:35

source

1 answer

how to split a very long string in python Create HTML table with c # from a datatable

score 1 · Answer 1

The tests, either CppUnit, GTest or any other usually have expressions such as CPPUNIT_ASSERT (cppunit) or ASSERT_TRUE (gtest) to evaluate certain results (what you want to test in the test).

In any case, the dynamics are the same ... if any check fails the test is given as not passed and no further checks are carried out.

In a way this behavior is not strange ... if you put a functionality in a test is to ensure that all the checks that make up the test are satisfactory ... it could also be dangerous to continue running the test:

void cppUnit_test::myTest()
{
  TestData* p = data;
  CPPUNIT_ASSERT(p != nullptr);

  while ( p->text != "END")
  {
    // ...
  }
}

Suppose that the check fails because the pointer is null ... does it make sense for the execution to continue and for the testing program to finish by trying to access p in while ?

If you want an evaluation for each chain you will have to program an independent test for each one.

Another possibility would be to evaluate the checks without using the test macros:

void cppUnit_test::myTest()
{
  TestData* p = data;

  int ok = 0;
  int total = 0;
  while ( p->text != "END")
  {
    std::stringstream ss;
    ss << "Text=\"" << p->text << "\" Counted=" << 
       countEnglishWords(p->text) << " Expected=" << p->englishWords;
    if(countEnglishWords(p->text) == p->englishWords )
      ok++;

    ++p;
    ++total;
  }

  CPPUNIT_ASSERT(ok==total);
}

But of course, then you have no information about which messages have not passed the validation