P$
P$

Controlling Indirect Inputs

Patrick Robertson
February 19, 2014

I'm going to talk a little bit today about indirect inputs in Ruby and how the relate to you as a tester of Ruby software. In a complicated system we'll have a ton of these things and it is important to understand some options we have to take control of them.

Defining and Framing Indirect Inputs

Let's take a simple example that is based upon when I start taking off my normal and boring pants and replacing them with more awesome and festive party pants:

class PartyTime
  def time_for_party_pants?
    Time.now.hour >= 23
  end
end

The reference to Time.now is what we in the fancy-names-for-simple-things industry like to refer to as an 'indirect input'. PartyTime cannot function without Time.now returning something that responds to a message of hour. It's an outgoing message that has side effects on the system we are directly working with.

Unfortunately for those of us who test applications, Time.now has a pretty reliable side effect: it returns the current time which in this case means that time_for_party_pants? only returns true one hour of the day. I'm not really a fan of only running the CI server for my app at 11pm and then having it pause itself till 12:01am in order to fully test out this method. Ain't nobody got time for that. So let's take control of that indirect input:

class PartyTime
  def time_for_party_pants?(now = Time.now)
    now.hour >= 23
  end
end

I can then test both true and false cases pretty easily:

class PartyPantsTest < MiniTest::Test
  def setup
    @party_time = PartyTime.new
  end

  def after_eleven_it_is_time_for_new_pants
    now = DateTime.parse('1/1/2001 11:01pm')
    assert true, @party_time.time_for_party_pants?(now)
  end

  def before_eleven_is_chino_time
    now = DateTime.parse('1/1/2001 9:14pm')
    assert false, @party_time.time_for_party_pants?(now)
  end
end

We can wield some very sharp instruments in Ruby to give us control over indirect inputs in our tests. Using a default parameter to represent the current time was probably the dullest, safest tool we can use. This is basically the only tool people who write in more static languages have available to them.

Instead, I can throw down with some pretty next level ruby shit:

class PartyTime
  def time_for_party_pants?
    current_time.hour >= 23
  end

  private
  def current_time
    Time.now
  end
end

class PartyPantsTest < MiniTest::Test
  def setup
    @party_time = PartyTime.new
  end

  def after_eleven_it_is_time_for_new_pants
    def @party_time.current_time
      DateTime.parse('1/1/2001 11:01pm')
    end

    assert true, @party_time.time_for_party_pants?(now)
  end

This is insanity! I redefined the private current_time method on PartyTime to return a specific time in the middle of my test suite. It only effects that one instance of PartyTime so we haven't destroyed the global behavior of PartyTime. There are developers in other languages that would kill for this tool being a part of their language. Unfortunately, this rock-hard awesome power can get out of hand quickly. Let's change the implementation of the PartyTime class in order to break everything:

class PartyTime
  def time_for_party_pants?
    current_time.hour >= 23
  end

  private
  def current_time_with_zone
    Time.with_zone.now
  end
end

Awwww shiiiiiiii. We changed the name of our private method and broke the implementation of time_for_party_pants?. Our test suite will pick that up right? rrrrrrrrrright?

Unfortunately, our test suite will pass because we've defined a method dynamically that ensures the test will pass. It doesn't actually check if the method existed prior to defining it, so we are essentially writing a test that will never, ever fail. This is less than ideal. We want our test suite to function as a canary in a coal mine. If we hose the interface of our class, we expect that canary to die.

I think we can now frame a couple guidelines for controlling indirect inputs given this contrived and trivial example (oh how I love to go from trivial examples to broad rules).

Test the interface of the indirect input

The previous example went terribly wrong when the interface of PartyTime and the testing stub diverged. There was nothing in place to ensure they both honored the same interface. This would've saved our bacon:

class PartyPantsTest < MiniTest::Test
  def setup
    @party_time = PartyTime.new
  end

  def after_eleven_it_is_time_for_new_pants
    raise "chirp" unless @party_time.respond_to?(:current_time)
    def @party_time.current_time
      DateTime.parse('1/1/2001 11:01pm')
    end

    assert true, @party_time.time_for_party_pants?(now)
  end

As it turns out, most ruby stubbing libraries are capable of acting like a canary in the coal mine if they are configured appropriately. Historically, I use Mocha paired with bourne to get the job done. This is what Mocha lets through by default:

  • Stubbing a method that is never called in a test
  • Stubbing a non-existant method
  • Stubbing a private/protected method

These are all things you probably want to blow up in your face so I'd suggest configuring them in such a way that doing any of those action results in an error message.

When we elect to create our own test doubles, we just need to build shared interface tests to protect ourselves. The example I provided above worked in a specific case but something like this is much more easily repeatable:

module PartyTimeInterfaceTest
  def responds_to_current_time
    assert true, @object.respond_to?(:current_time)
  end
end

class PartyPantsTest < MiniTest::Test
  include PartyTimeInterfaceTest

  def setup
    @party_time = @object = PartyTime.new
  end

  def after_eleven_it_is_time_for_new_pants
    def @party_time.current_time
      DateTime.parse('1/1/2001 11:01pm')
    end

    assert true, @party_time.time_for_party_pants?(now)
  end

If we needed to build out a whole fake class that represented the interface, we could include the PartyTimeInterfaceTest module and be confident we're safe. This allows us to breathe a bit easier in the very dynamic world of Ruby.

Prefer to inject indirect inputs

The pattern that gives us the most flexibility around controlling indirect inputs actually ends up being one of the five pillars of SOLID design (Dependency Inversion). When PartyTime collaborates with any old object that responds to hour we are freed from worrying about implementation details. When it is tied directly to Time.now.hour we can only deal with the system time.

Examples of testing with injection

I've got some pretty good testing options with the former choice. We've got a lot of flexibility in what we can use in Ruby for injectable test doubles because we're rolling with the most dynamic, duck-type loving language around.

Inject the intended return object

The easiest way to go is to just inject an instance of the object you're expecting to work with in the method:

class PartyPantsTest < MiniTest::Test
  def setup
    @party_time = PartyTime.new
  end

  def after_eleven_it_is_time_for_new_pants
    now = DateTime.parse('1/1/2001 11:01pm')
    assert true, @party_time.time_for_party_pants?(now)
  end

I'd use this when the message hour isn't considered expensive or slow on the concept of now. We can control exactly what hour of the day it is in the tests and our intention in the method is clear.

Returning a fake object

Instead, we can just build ourselves an object that responds to the interface and returns something reasonable to test logic within our own class. OpenStruct is pretty good for this sort of thing:

require 'ostruct'
class PartyPantsTest < MiniTest::Test
  def setup
    @party_time = PartyTime.new
  end

  def after_eleven_it_is_time_for_new_pants
    now = OpenStruct.new(hour: 23)
    assert true, @party_time.time_for_party_pants?(now)
  end

If I consider the hour method on a DateTime object to be slow and/or costly I can choose to remove it from the equation altogether.

Both these examples function pretty well as a canary in the coal mine. The latter example will break more often than the former example if you change the details of collaboration with now. If you decide that you should look at the minutes instead of the hour the fake will fall apart but the DateTime injection should be OK.

These examples are also pretty clear about the implementation of the system. We know what sort of contract our indirect inputs are honoring and we're not bogged down in the details of additional libraries.

Examples of relying on concrete classes

If we choose to rely directly on the concrete example of Time.now we've got a couple reasonable options for controlling it as indirect input as well.

Stub the class method

class PartyPantsTest < MiniTest::Test
  def setup
    @party_time = PartyTime.new
  end

  def after_eleven_it_is_time_for_new_pants
    now = DateTime.parse('1/1/2001 11:01pm')
    Time.stubs(:now).returns(now)

    assert true, @party_time.time_for_party_pants?
  end

Earlier in the post, I demonstrated how straightforward it is to stub out instance methods. Stubbing a class method is a much more delicate affair because we need to undo the action almost immediately to return the class to its normal behavior. Otherwise we end up changing the behavior of each subsequent test. You can do it by setting up method aliases, but it's just more easily left to libraries that are dedicated to doing it.

You run the risk here that Time stops responding to .now and that your test suite will pass when your actual code breaks. It's pretty safe to do this for Core classes, but if I was stubbing something that I owned/had tests for I'd include a test that ensures the class responds to that interface.

Use a time manipulation library

Another method you can utilize is sort of a 'grand stub'. In the case of Time, that would be a library like Timecop that has the ability to freeze and unfreeze time at will. It looks a bit like this:

class PartyPantsTest < MiniTest::Test
  def setup
    @party_time = PartyTime.new
  end

  def after_eleven_it_is_time_for_new_pants
    Timecop.freeze('1/1/2001 11:01pm')

    assert true, @party_time.time_for_party_pants?

    Timecop.unfreeze
  end

This is a pretty common testing strategy. I typically freeze time before the test suite even begins running and then unfreeze it once it stops. On tests that rely directly on time, we still need to be pretty explicit about what we're doing. If Time was frozen to an hour after 11pm by default and I relied upon that in the above test, another developer would have no clue why my test magically passes without those Timecop method calls.

Both of these examples seem to come with side effects that I'm not a huge fan of. The former example needs to provide some sort of guarantee that Time actually honors the interface that it is stubbing. We can either write additional tests (that sometimes look smell like overspecification to another developer) or ensure our stubbing library won't allow a stub to work on a non-defined method. The latter example has a slightly more involved setup/teardown requirement to ensure that people viewing the tests are aware of what's going on.

Wrapping up

Gaining control over indirect inputs is a pretty necessary reality for anyone testing Ruby code. We have a pretty wide range of tools available to us, some due to the nature of the dynanicism of Ruby and others that have been handed down from the likes of jUnit.

Understanding the trade-offs in system complexity, readability of tests, and how/why the system being executed may differ from the test suite is a pretty important skill. I've personally found that being able to directly inject indirect inputs to be a valuable pattern in my time writing Ruby code and believe that you may too!

comments powered by Disqus