When working on Refactoring, Ruby Edition, I realised that Separate Query From Modifier was one of my favourite refactorings. I'd been doing it for a while without realising that it had a formal name. I probably hadn't realised the full benefits of it either. From Refactoring, Ruby Edition: "When you have a function that gives you a value and has no observable side effects, you have a very valuable thing. You can call this function as often as you like. You can move the call to other places in the method. In short, you have a lot less to worry about." If you do not separate the querying code from the modifying code, code becomes difficult to understand and re-use. When I'm trying to track down a bug, I'm looking for two things: 1. The code that triggers the offending code (the query), and 2. the offending code itself (the modifier). If I have to search through a method that makes a query, then does some modification, then does another query, and some more modification, then my head starts to hurt. Which query returns the result that triggers the bug? And which modification is the bad modification? If we can separate the querying code from the modifying code, we can often achieve a better abstraction of our business rules and promote re-use. As a project evolves, we often have to introduce new trigger points for state changes. And in an agile environment, I often see the story cards evolve like this:
Story 1: Under condition 'X', 'A' should change such that...
Story 2: Under condition 'X', warn the user before making the change to 'A'
As Agile developers, we like this kind of story breakdown. After completing story 1, we can demonstrate to the user that we understand condition 'X' and the changes that should be made to 'A'. If we get it wrong, then we can fix it. Story 2 is the icing on the cake. It provides some nicer usability around the feature. It might also be a lower priority, and we might be able to release the code without Story 2 and gain some real business value before we polish it later. So the separation of the stories could be important. But if we mix query and modifier, it can become very difficult to introduce the warning, or introduce new trigger points for the desired state change.
But I've realised that Separate Query From Modifier is not always as easy as extracting conditional logic to one method, and having the modifying logic in another method. In my last post I said that in Rails, it can sometimes be difficult to re-wire multiple ActiveRecord objects of different type together according to some set of business rules without using the database as a storage mechanism, and without the validations getting in your way. We've often ended up with complex service methods that perform a query, do some modification (saving to the database), perform another query, do some more modification, and so on. You might end up with 5 or 6 queries that have to be performed in sequence. Later queries might depend on modifications that have been performed as a result of earlier queries. And the code is ugly, and difficult to re-use.
One way that we've solved this problem is to create a results object that represents the new relationships to be created. It's just a plain old Ruby object with some attributes. Let's say we're trying to build a new 'A' object, with relationships to B and C. Let's say A has_many B, and A has_one C. I'd create a results object called NewA, with an array attribute for the Bs and an attribute for the C. As I go through my algorithm, I can add my Bs and my C to the results object - without actually changing any underlying associations. (I now have a query without the modifier). Toward the end of the algorithm, I can present my results object to the user for confirmation, and if they confirm the change, then I can grab my results object and make the actual associations in the database (the modifier). I might even find that I can move some behaviour to this results object, and it will cease being a dumb data object. But even if I don't get to move any behaviour, the separation of query and modifier is worth the effort.
Sunday, January 27, 2008
Friday, January 25, 2008
The evolution of a Domain in rails: Part 1
I've been at my current client for about 15 months now, and during that time we've been working in fairly complex domains. Whilst our application started out very CRUD-like, it gradually moved away from CRUD in its simplest form (resources with simple attributes) to quite complex domain models. We now have quite a few functions that cause the interaction of 10-plus domain objects and often require the re-wiring of these objects according to a change in state triggered by a user. I was recently asked by a colleague to explain the limitations of ActiveRecord and some ways that we've overcome those issues. His concern was that Rails' tight coupling to the database would lead to anaemic domain objects that are effectively just DTO's, with all of the business logic creeping up into the view.
We've never ended up with business logic in the view, but we have made mistakes such that this behaviour has ended up in non-ideal parts of the application. Our first mistake was to place too much business logic in the controllers. This came about when we were trying to orchestrate the interactions between (say) three domain objects of different type. We asked ourselves "This behaviour doesn't belong on any one of the three domain objects, so where should it go?". Rails makes a clear distinction between Model, View, and Controller, and most of the examples only show "model" objects as being those that inherit from ActiveRecord::Base (and are therefore stored in a database). So given that we didn't want this business logic in the view, and it didn't belong on any of the "model" objects that we already had, the only place left was the controller. So our controllers got fat. And they were hard to test. And hard to change. And the business logic was very hard to re-use. And our model objects were thin (you might even say anaemic). Boo.
Fresh from reading Domain Driven Design, our teammate Pat Sarnacke came to the rescue, saying "We need services". And it's interesting to note that we were not alone in this discovery. So we extracted the business logic from our complex controller actions into Service objects. (From Domain Driven Deisgn, service objects are objects "with no state of their own nor any meaning in the domain beyond the operation they host"). We got quite a bit of re-use of this logic by doing so. It was definitely a step in the right direction. But then our Services started to grow. They became large and complex, and although they displayed the positive trait of having low coupling (with perhaps only one public method), they weren't very cohesive. We extracted private methods that were cohesive within themselves, but the collection of private methods made no sense together, other than the fact that they were part of the "procedure" of the service. And so the service became hard to understand in isolation, and not particularly amenable to re-use. If you wanted to re-use the service as a whole, then you were in good shape (much better than when the logic was in the controller). But if you wanted to re-use one of the private methods, it wasn't so easy (and not simply because they were private).
It turns out that, in some cases, it can be difficult to re-wire multiple ActiveRecord objects of different type together according to some set of business rules without using the database as a temporary storage mechanism. We would often end up with service classes that would first associate model A to model B, save model A, and then query model B for some further modifications to be performed. We'd have to reload model B in order to make this query (because it depended on the relationship to A). And so our services ended up being a sprinkling of save!s and reloads. And this was not because it was impossible to perform this logic in any other way, but because it was much easier to make the association, save it, and then go on our merry way through the rest of the algorithm.
And this worked for a little while. But any sub-section of the service was almost impossible to re-use. The order of association became very fragile because of our validations - you'd have to disassociate C from A before associating B to A because A couldn't have both B and C. And perhaps C needed a replacement for A to be valid, so you'd have to find C a new 'A' before you could save C. And so not only was the service procedural, but it was strictly procedural - you had to perform each step in a defined order. This made re-use very difficult. And what we really wanted to do was to move some of the logic onto domain objects, but with all the saving and reloading that was going on, it seemed almost impossible to extract logic that made any sense outside of the context of our complex algorithm.
And the killer came when we were asked to warn the user before we made this complex change, and give them information about the change that was about to be made so that they could decide whether or not to proceed. In order to know whether the change was going to be made, and what the change would look like, we'd have to traverse the entire algorithm, by which time the change would have been made. In short, we couldn't separate the querying code from the modifying code (see Separate Query from Modifier in Refactoring). This was the reason we couldn't fulfill the feature, and the reason that we couldn't move cohesive logic to the domain objects.
It took us two steps to solve these two problems (fragility and Separate Query From Modifier). I'll describe the solution to the fragility here, and the solution to Separate Query from Modifier in the next post.
Our validations started to dictate the order in which our objects had to be saved (and therefore the order of the steps in our algorithm), which made re-use of code very difficult. So we decided that we needed a way to save without triggering the validations, but ensure that only valid objects were left in the database after the algorithm had finished. ActiveRecord provides a method called save_without_validation that takes care of our first requirement. And if we called save_without_validation within a transaction, and then raised an Exception if any of the objects were invalid at the end of the algorithm, then the transaction would roll back, and we wouldn't have invalid records in the database.
So we added a method called save_and_record_without_validation to ActiveRecord::Base which calls save_without_validation on the object, and stores the object in the Thread.current hash so that we can go back at the end of the algorithm and ensure that it is valid. We have a method called validate_recorded_records! which takes a block that contains the calls to save_and_record_without_validation, and then validates after the block has been executed. The calling code might look something like this:
And if line 10 causes an invalid object until line 11 is executed, then it doesn't matter, because the validation only gets performed after the validate_recorded_records! block has been executed. The source code for our RecordingHelper can be found here and the tests here
So this solved our fragility problem - the order of execution of the statements no longer mattered as much, which enabled us to extract methods that could be re-used. But it still doesn't separate query from modifer - we'll tackle that in the next post.
We've never ended up with business logic in the view, but we have made mistakes such that this behaviour has ended up in non-ideal parts of the application. Our first mistake was to place too much business logic in the controllers. This came about when we were trying to orchestrate the interactions between (say) three domain objects of different type. We asked ourselves "This behaviour doesn't belong on any one of the three domain objects, so where should it go?". Rails makes a clear distinction between Model, View, and Controller, and most of the examples only show "model" objects as being those that inherit from ActiveRecord::Base (and are therefore stored in a database). So given that we didn't want this business logic in the view, and it didn't belong on any of the "model" objects that we already had, the only place left was the controller. So our controllers got fat. And they were hard to test. And hard to change. And the business logic was very hard to re-use. And our model objects were thin (you might even say anaemic). Boo.
Fresh from reading Domain Driven Design, our teammate Pat Sarnacke came to the rescue, saying "We need services". And it's interesting to note that we were not alone in this discovery. So we extracted the business logic from our complex controller actions into Service objects. (From Domain Driven Deisgn, service objects are objects "with no state of their own nor any meaning in the domain beyond the operation they host"). We got quite a bit of re-use of this logic by doing so. It was definitely a step in the right direction. But then our Services started to grow. They became large and complex, and although they displayed the positive trait of having low coupling (with perhaps only one public method), they weren't very cohesive. We extracted private methods that were cohesive within themselves, but the collection of private methods made no sense together, other than the fact that they were part of the "procedure" of the service. And so the service became hard to understand in isolation, and not particularly amenable to re-use. If you wanted to re-use the service as a whole, then you were in good shape (much better than when the logic was in the controller). But if you wanted to re-use one of the private methods, it wasn't so easy (and not simply because they were private).
It turns out that, in some cases, it can be difficult to re-wire multiple ActiveRecord objects of different type together according to some set of business rules without using the database as a temporary storage mechanism. We would often end up with service classes that would first associate model A to model B, save model A, and then query model B for some further modifications to be performed. We'd have to reload model B in order to make this query (because it depended on the relationship to A). And so our services ended up being a sprinkling of save!s and reloads. And this was not because it was impossible to perform this logic in any other way, but because it was much easier to make the association, save it, and then go on our merry way through the rest of the algorithm.
And this worked for a little while. But any sub-section of the service was almost impossible to re-use. The order of association became very fragile because of our validations - you'd have to disassociate C from A before associating B to A because A couldn't have both B and C. And perhaps C needed a replacement for A to be valid, so you'd have to find C a new 'A' before you could save C. And so not only was the service procedural, but it was strictly procedural - you had to perform each step in a defined order. This made re-use very difficult. And what we really wanted to do was to move some of the logic onto domain objects, but with all the saving and reloading that was going on, it seemed almost impossible to extract logic that made any sense outside of the context of our complex algorithm.
And the killer came when we were asked to warn the user before we made this complex change, and give them information about the change that was about to be made so that they could decide whether or not to proceed. In order to know whether the change was going to be made, and what the change would look like, we'd have to traverse the entire algorithm, by which time the change would have been made. In short, we couldn't separate the querying code from the modifying code (see Separate Query from Modifier in Refactoring). This was the reason we couldn't fulfill the feature, and the reason that we couldn't move cohesive logic to the domain objects.
It took us two steps to solve these two problems (fragility and Separate Query From Modifier). I'll describe the solution to the fragility here, and the solution to Separate Query from Modifier in the next post.
Fragility
Our validations started to dictate the order in which our objects had to be saved (and therefore the order of the steps in our algorithm), which made re-use of code very difficult. So we decided that we needed a way to save without triggering the validations, but ensure that only valid objects were left in the database after the algorithm had finished. ActiveRecord provides a method called save_without_validation that takes care of our first requirement. And if we called save_without_validation within a transaction, and then raised an Exception if any of the objects were invalid at the end of the algorithm, then the transaction would roll back, and we wouldn't have invalid records in the database.
So we added a method called save_and_record_without_validation to ActiveRecord::Base which calls save_without_validation on the object, and stores the object in the Thread.current hash so that we can go back at the end of the algorithm and ensure that it is valid. We have a method called validate_recorded_records! which takes a block that contains the calls to save_and_record_without_validation, and then validates after the block has been executed. The calling code might look something like this:
1 old_parent = Parent.find_by_name("oldie")
2 new_parent = Parent.find_by_name("newbie")
3 new_child = Child.find(5)
4 old_child = old_parent.child
5 old_child.parent = new_parent
6 old_parent.child = new_child
7
8 Parent.transaction do
9 validate_recorded_records! do
10 old_child.save_and_record_without_validation
11 old_parent.save_and_record_without_validation
12 end
13 end
14
15
And if line 10 causes an invalid object until line 11 is executed, then it doesn't matter, because the validation only gets performed after the validate_recorded_records! block has been executed. The source code for our RecordingHelper can be found here and the tests here
So this solved our fragility problem - the order of execution of the statements no longer mattered as much, which enabled us to extract methods that could be re-used. But it still doesn't separate query from modifer - we'll tackle that in the next post.
Subscribe to:
Posts (Atom)
