Controlling Indirect Inputs
I'm going to talk a little bit today about indirect inputs in Ruby and how the relate to you as a tester of Ruby software. In a complicated system we'll have a ton of these things and it is important to understand some options we have to take control of them.
Defining and Framing Indirect Inputs
Let's take a simple example that is based upon when I start taking off my normal and boring pants and replacing them with more awesome and festive party pants:
class PartyTime def time_for_party_pants? Time.now.hour >= 23 end end
The reference to
Time.now is what we in the fancy-names-for-simple-things industry like to refer to as an 'indirect input'.
PartyTime cannot function without
Time.now returning something that responds to a message of
hour. It's an outgoing message that has side effects on the system we are directly working with.
Unfortunately for those of us who test applications,
Time.now has a pretty reliable side effect: it returns the current time which in this case means that
time_for_party_pants? only returns
true one hour of the day. I'm not really a fan of only running the CI server for my app at 11pm and then having it pause itself till 12:01am in order to fully test out this method. Ain't nobody got time for that. So let's take control of that indirect input:
class PartyTime def time_for_party_pants?(now = Time.now) now.hour >= 23 end end
I can then test both true and false cases pretty easily:
class PartyPantsTest < MiniTest::Test def setup @party_time = PartyTime.new end def after_eleven_it_is_time_for_new_pants now = DateTime.parse('1/1/2001 11:01pm') assert true, @party_time.time_for_party_pants?(now) end def before_eleven_is_chino_time now = DateTime.parse('1/1/2001 9:14pm') assert false, @party_time.time_for_party_pants?(now) end end
We can wield some very sharp instruments in Ruby to give us control over indirect inputs in our tests. Using a default parameter to represent the current time was probably the dullest, safest tool we can use. This is basically the only tool people who write in more static languages have available to them.
Instead, I can throw down with some pretty next level ruby shit:
class PartyTime def time_for_party_pants? current_time.hour >= 23 end private def current_time Time.now end end class PartyPantsTest < MiniTest::Test def setup @party_time = PartyTime.new end def after_eleven_it_is_time_for_new_pants def @party_time.current_time DateTime.parse('1/1/2001 11:01pm') end assert true, @party_time.time_for_party_pants?(now) end
This is insanity! I redefined the private
current_time method on
PartyTime to return a specific time in the middle of my test suite. It only effects that one instance of
PartyTime so we haven't destroyed the global behavior of
PartyTime. There are developers in other languages that would kill for this tool being a part of their language. Unfortunately, this rock-hard awesome power can get out of hand quickly. Let's change the implementation of the
PartyTime class in order to break everything:
class PartyTime def time_for_party_pants? current_time.hour >= 23 end private def current_time_with_zone Time.with_zone.now end end
Awwww shiiiiiiii. We changed the name of our private method and broke the implementation of
time_for_party_pants?. Our test suite will pick that up right? rrrrrrrrrright?
Unfortunately, our test suite will pass because we've defined a method dynamically that ensures the test will pass. It doesn't actually check if the method existed prior to defining it, so we are essentially writing a test that will never, ever fail. This is less than ideal. We want our test suite to function as a canary in a coal mine. If we hose the interface of our class, we expect that canary to die.
I think we can now frame a couple guidelines for controlling indirect inputs given this contrived and trivial example (oh how I love to go from trivial examples to broad rules).
Test the interface of the indirect input
The previous example went terribly wrong when the interface of
PartyTime and the testing stub diverged. There was nothing in place to ensure they both honored the same interface. This would've saved our bacon:
class PartyPantsTest < MiniTest::Test def setup @party_time = PartyTime.new end def after_eleven_it_is_time_for_new_pants raise "chirp" unless @party_time.respond_to?(:current_time) def @party_time.current_time DateTime.parse('1/1/2001 11:01pm') end assert true, @party_time.time_for_party_pants?(now) end
As it turns out, most ruby stubbing libraries are capable of acting like a canary in the coal mine if they are configured appropriately. Historically, I use Mocha paired with bourne to get the job done. This is what Mocha lets through by default:
- Stubbing a method that is never called in a test
- Stubbing a non-existant method
- Stubbing a private/protected method
These are all things you probably want to blow up in your face so I'd suggest configuring them in such a way that doing any of those action results in an error message.
When we elect to create our own test doubles, we just need to build shared interface tests to protect ourselves. The example I provided above worked in a specific case but something like this is much more easily repeatable:
module PartyTimeInterfaceTest def responds_to_current_time assert true, @object.respond_to?(:current_time) end end class PartyPantsTest < MiniTest::Test include PartyTimeInterfaceTest def setup @party_time = @object = PartyTime.new end def after_eleven_it_is_time_for_new_pants def @party_time.current_time DateTime.parse('1/1/2001 11:01pm') end assert true, @party_time.time_for_party_pants?(now) end
If we needed to build out a whole fake class that represented the interface, we could include the
PartyTimeInterfaceTest module and be confident we're safe. This allows us to breathe a bit easier in the very dynamic world of Ruby.
Prefer to inject indirect inputs
The pattern that gives us the most flexibility around controlling indirect inputs actually ends up being one of the five pillars of SOLID design (Dependency Inversion). When
PartyTime collaborates with any old object that responds to
hour we are freed from worrying about implementation details. When it is tied directly to
Time.now.hour we can only deal with the system time.
Examples of testing with injection
I've got some pretty good testing options with the former choice. We've got a lot of flexibility in what we can use in Ruby for injectable test doubles because we're rolling with the most dynamic, duck-type loving language around.
Inject the intended return object
The easiest way to go is to just inject an instance of the object you're expecting to work with in the method:
class PartyPantsTest < MiniTest::Test def setup @party_time = PartyTime.new end def after_eleven_it_is_time_for_new_pants now = DateTime.parse('1/1/2001 11:01pm') assert true, @party_time.time_for_party_pants?(now) end
I'd use this when the message
hour isn't considered expensive or slow on the concept of
now. We can control exactly what hour of the day it is in the tests and our intention in the method is clear.
Returning a fake object
Instead, we can just build ourselves an object that responds to the interface and returns something reasonable to test logic within our own class.
OpenStruct is pretty good for this sort of thing:
require 'ostruct' class PartyPantsTest < MiniTest::Test def setup @party_time = PartyTime.new end def after_eleven_it_is_time_for_new_pants now = OpenStruct.new(hour: 23) assert true, @party_time.time_for_party_pants?(now) end
If I consider the
hour method on a
DateTime object to be slow and/or costly I can choose to remove it from the equation altogether.
Both these examples function pretty well as a canary in the coal mine. The latter example will break more often than the former example if you change the details of collaboration with
now. If you decide that you should look at the minutes instead of the hour the fake will fall apart but the
DateTime injection should be OK.
These examples are also pretty clear about the implementation of the system. We know what sort of contract our indirect inputs are honoring and we're not bogged down in the details of additional libraries.
Examples of relying on concrete classes
If we choose to rely directly on the concrete example of
Time.now we've got a couple reasonable options for controlling it as indirect input as well.
Stub the class method
class PartyPantsTest < MiniTest::Test def setup @party_time = PartyTime.new end def after_eleven_it_is_time_for_new_pants now = DateTime.parse('1/1/2001 11:01pm') Time.stubs(:now).returns(now) assert true, @party_time.time_for_party_pants? end
Earlier in the post, I demonstrated how straightforward it is to stub out instance methods. Stubbing a class method is a much more delicate affair because we need to undo the action almost immediately to return the class to its normal behavior. Otherwise we end up changing the behavior of each subsequent test. You can do it by setting up method aliases, but it's just more easily left to libraries that are dedicated to doing it.
You run the risk here that
Time stops responding to
.now and that your test suite will pass when your actual code breaks. It's pretty safe to do this for Core classes, but if I was stubbing something that I owned/had tests for I'd include a test that ensures the class responds to that interface.
Use a time manipulation library
Another method you can utilize is sort of a 'grand stub'. In the case of
Time, that would be a library like Timecop that has the ability to freeze and unfreeze time at will. It looks a bit like this:
class PartyPantsTest < MiniTest::Test def setup @party_time = PartyTime.new end def after_eleven_it_is_time_for_new_pants Timecop.freeze('1/1/2001 11:01pm') assert true, @party_time.time_for_party_pants? Timecop.unfreeze end
This is a pretty common testing strategy. I typically freeze time before the test suite even begins running and then unfreeze it once it stops. On tests that rely directly on time, we still need to be pretty explicit about what we're doing. If Time was frozen to an hour after 11pm by default and I relied upon that in the above test, another developer would have no clue why my test magically passes without those
Timecop method calls.
Both of these examples seem to come with side effects that I'm not a huge fan of. The former example needs to provide some sort of guarantee that
Time actually honors the interface that it is stubbing. We can either write additional tests (that sometimes look smell like overspecification to another developer) or ensure our stubbing library won't allow a stub to work on a non-defined method. The latter example has a slightly more involved setup/teardown requirement to ensure that people viewing the tests are aware of what's going on.
Gaining control over indirect inputs is a pretty necessary reality for anyone testing Ruby code. We have a pretty wide range of tools available to us, some due to the nature of the dynanicism of Ruby and others that have been handed down from the likes of jUnit.
Understanding the trade-offs in system complexity, readability of tests, and how/why the system being executed may differ from the test suite is a pretty important skill. I've personally found that being able to directly inject indirect inputs to be a valuable pattern in my time writing Ruby code and believe that you may too!