RubyFIT

Framework for Integrated Tests in Ruby

The examples in this document are taken from the FitBook by Rick Mugridge and Ward Cunningham.

Download the code bundle for the basic and this advanced tutorials.

Using other values in tables

Basic FIT examples usually involves tables containing simple values, such as integer or floating point numbers and strings, which are mapped in fixtures onto standard types such as Fixnum, Float and String. This is typically done in order to keep examples as simple as possible, but sometimes it comes at the cost of sacrificing part of the conceptual integrity of the application’s domain.

For instance, when you are calculating a provided discount on a purchase amount, domain experts would talk about money, not just some floating point numbers. A properly crafted application would use the right abstractions to represent fundamental domain objects, and tests would exercise the application using those same abstractions. In our case study, that would dictate the creation and use of a Money class, to be provided with a reasonable specification of the format used to display its values. In the following table, taken from page 214 in the FitBook, money values are shown with a leading $ and exactly two decimal places of precision.

Even if the Ruby code for the CalculateDiscountMoney fixture could be identical to the code for the CalculateDiscount fixture in the simpler example which used floating point numbers instead of money values, the type of the instance variable amount and the type returned by the discount method would now be Money instead of Float, a type which the FIT framework does not handle automatically.

Quoting the FitBook, page 215: “To handle conversion from the textual form of the value to a corresponding object (here, an instance of Money) the fixture defines the method parse. This method overrides the default method in the Fixture class to specify how to convert a textual value from a table cell into an object of the right class.”

The implementation pattern for the parse method is always the same. It is passed a string, containing the textual value to be converted into an object, and a class, representing the type which the conversion must happen towards. The method checks whether the type is one of those types the fixture has to deal with; if so, it asks for the conversion to the same class representing that type; otherwise, the superclass method is called to handle the task.

class CalculateDiscountMoney < Fit::ColumnFixture
  # ...
  def parse string, klass
    return Money.parse(string) if klass == Money
    super
  end
end

But how the right klass value could be passed to the parse method, given that Ruby is a dynamically typed language, and that no information about the type of fields and the return type of methods is available just from the structure of the code, without necessarily running it? While RubyFIT is able to recognize and handle basic standard types as numbers, strings, booleans, it needs additional support from fixture programmers when it has to handle custom types, such as the Money class in our example.

A static metadata structure has to be provided in the fixture. It takes the form of a dictionary where the keys are strings containing the name of fields and methods as they are written in the HTML table, and values are classes representing the type of those fields or the return type of those methods. In our case study, the metadata for the CalculateDiscountMoney fixture would take the following shape.

class CalculateDiscountMoney < Fit::ColumnFixture
  @@metadata = { 'amount' => Money, 'discount()' => Money }
  # ...
end

Then, coming across cells in the HTML table, the framework will be able to know the right type of the data contained in those cells, and to ask the fixture for the proper conversion. See also the FrequentlyAskedQuestions for a comparison of the same mechanism in a similar example between dynamically typed languages such as Ruby and statically typed languages such as Java or C#.

Grouping fixtures into modules

The FitBook contains a longish case study on a fictitious application called RentEz, a software for managing the rental of party equipment, such as party tents. The case study is focussed “on how FIT tests are created and evolve and how they can be incorporated into the software development process of an organization.” The first business rule to be tested concerns the application of fees for late returns of equipment items. As it may happen in real world, nontrivial applications, the rule is quite complicated.

A late fee may be charged if equipment is expected to be returned by a certain time. The fee depends on:

• How long the item was rented for. There is usually one hour’s grace. However, the longer the time, the longer the grace period.
• If the return is beyond the grace period, the late fee is based on either the extra time or the extra time minus the grace period.
• Usually, the late fee is simply the extra time, as defined, charged at the usual rate for that item.
• However, when the item is in high demand, the grace period is reduced and an extra late fee is charged. This is especially important for rental items that have been booked out, as the rental company staff would need to source replacement equipment from elsewhere at some cost.

The rules stated above are captured in the following HTML table, taken from the FitBook, page 99.

Since the RentEz software would be wide and complex, it has been chosen to separate test code from application code modularizing it in a separate package, or in Ruby parlance, a separate module. An evidence of the modularization can be seen in the fixture name contained in the first row of the HTML table: it is indeed composed of two names separated by a dot. The first name is the name of the package, or module; the second name is the name of the fixture contained in that module. In the Ruby world, the module name in the HTML table is translated into the name of a Ruby module just capitalizing its first letter. The fixture name remains unchanged. So, from an HTML table header containing rent.CalculateLateHours, programmers would extract a RubyFIT fixture called Rent::CalculateLateHours with the following skeleton.

require 'fit/column_fixture'

module Rent
  class CalculateLateHours < Fit::ColumnFixture
  end
end

How about file system mappings? Where do programmers have to write that code? General fixtures rules apply: the name of the fixture is also the name of the file containing the fixture, where the camel cased words are lied without capital letters and separated by underscores. The above code snippet would then be placed in a file called calculate_late_hours.rb. But when modules get introduced, one more rule applies: the file must be contained in a subdirectory, typically relative to the directory where you are invoking the framework executable scripts, named after the uncapitalized module name. In our case, the calculate_late_hours.rb file would need to be included in a subdirectory called rent.

To clarify the new rule, let's suppose to be working in the $HOME directory, and that our working directory contains the TestLateHours.html file with the HTML table including test data. Then, we typically would need to create a $HOME/rent subdirectory, and place there the calculate_late_hours.rb file containing the fixture included in the Rent module. We could finally execute the following command from our working directory:

fit TestLateHours.html ReportLateHours.html

and have RubyFIT correctly finding the fixture, running the test and writing a report in the specified file in our $HOME directory.

Using the Rake FitTask task

As more and more tests get written, means to achieve a more automated workflow other than a command line executable script will be needed. Rake is a build tool with capabilities similar to the make program, but which uses standard Ruby code to declare tasks and dependencies. Widely deployed in the Ruby world, and one of the most downloaded program according to gems statistics and projects statistics on RubyForge, Rake seems to be the perfect framework for writing a tool to automate the running and checking of RubyFIT test cases. As a result, enter a new Rake task called FitTask.

The first thing you would be able to do is to run tests written using the RubyFIT framework. In a new Rakefile (which can be named Rakefile or rakefile, with or without the rb extension) you just need to write the following code:

require 'fittask'

Rake::FitTask.new(:fit) do |t|
  # modify t to add tests...
end

to define a couple of new tasks aimed at running FIT tests. In fact, running rake -T in the same directory containing the Rakefile will result in the following list of tasks.

rake fit         # Run FIT acceptance tests
rake fit_report  # Run FIT acceptance tests with HTML reports

The first task, named after the string corresponding to the symbol passed as a parameter to the FitTask constructor, runs the tests typically defined in the closure which the constructor passes the new task to. The second task, named after the first task plus a _report suffix, runs the same tests as the first task, only this time producing the HTML reports from the data sources.

A single test is represented by a dictionary composed of five entries: the name of the test, corresponding to the name of the HTML file containing the data, and the figures for the right, wrong, ignores and exceptions expected results. Of these five components, only the first is mandatory, while the other four are optional. For instance, we could write the following entry for the TestDiscount example.

Rake::FitTask.new(:fit) do |t|
  test = { :name => 'TestDiscount',
           :right => 7, :wrong => 1,
           :ignores => 0, :exceptions => 0 }
  # ...
end

There are some tests which contain errors (or exceptions, or situations where the data have to be ignored) on purpose, like this TestDiscount example, or the MusicExampleWithErrors from the FIT official wiki. In these cases, the chance of specifying exactly the amounts for the different outcomes from the test comes very handy. When the test runs through Rake, the task will check that the right, wrong, ignores and exceptions results exactly amount at the quantity specified in the dictionary, raising an error otherwise.

If, on the other hand, every case in a test has to be successful, that test can be represented using its name only. When running Rake, the task will raise an error if the test is found to have at least a wrong, ignored, or exceptional case.

Rake::FitTask.new(:fit) do |t|
  test = { :name => 'TestDiscount',
           :right => 7, :wrong => 1,
           :ignores => 0, :exceptions => 0 }
  successful_test = { :name => 'TestLateHours' }
  # ...
end

To be run, tests must be grouped in suites in the following fashion.

Rake::FitTask.new(:fit) do |t|
  t.create_test_suite do |suite|
    test = { :name => 'TestDiscount',
             :right => 7, :wrong => 1,
             :ignores => 0, :exceptions => 0 }
    successful_test = { :name => 'TestLateHours' }
    suite.tests = [test, successful_test]
  end
end

Other properties for the suite variable can be set: a test_path and a report_path can be specified (defaulted to the current directory) but for our examples we will assume that HTML source files are placed in the same directory as the Rakefile, and that HTML reports will be produced in that very same directory. Besides, using the fail_on_failed_test boolean flag (defaulted to false) you can also decide if the Rake execution has to be stopped whenever an error with respect to the data specified in the Rakefile is found, or the execution must be carried on anyway.

Now, running rake fit from the command line will execute the two tests in the suite, displaying the file name currently used, and reporting no error because the run will be successfully follows the results specified in the Rakefile. If you run rake fit_report, HTML file reports will also be generated, using the same name of the HTML file containing the orignal data with an added Report_ prefix.

To further refine the Rakefile in order to run all the tests contained in the basic and advanced examples of RubyFIT usage, two suites can be created, logically separating cases when some errors have to occur from those that have to be completely successful.

Rake::FitTask.new(:fit) do |t|
  t.create_test_suite do |suite|
    tests = []
    tests << { :name => 'TestDiscount',
               :right => 7, :wrong => 1, :ignores => 0, :exceptions => 0 }
    tests << { :name => 'TestDiscountMoney',
               :right => 7, :wrong => 1, :ignores => 0, :exceptions => 0 }
    suite.tests = tests
  end
  t.create_test_suite do |suite|
    tests = []
    tests << { :name => 'TestChatServer' }
    tests << { :name => 'TestDiscountGroup' }
    tests << { :name => 'TestLateHours' }
    suite.tests = tests
  end
end

Finally, since reports generate new HTML files, you may want to automatically get rid of them, using the functionalities already built in Rake. The usage pattern from within the FitTask task typically takes form along the following lines.

require 'rake/clean'

Rake::FitTask.new(:fit) do |t|
  # test suites are already set up...
  t.test_suites.each { |suite| CLOBBER.include('Report_*.html') }
end

Calling rake clobber from the command line after running the fit_report task will result in the removal of the included generated files.