AcidState - Best Practices?

[mifrai]

Hi,

I have a few questions about AcidState best practices / behavior and I
was hoping to get some help. In particular, I'm wondering about the
following things:

* Should I use a single monolithic AcidState or should l use several
smaller ones? What are the advantages of one vs. the other?

I have a few guesses, but I'm not too familiar with AcidState - so
please correct me if I'm wrong. One advantage of a single monolithic
one that I can think of is that you can get transactions across every
piece of data in your code. A disadvantage is that all writes must
happen sequentially and thus you're essentially introducing a global
lock on your database writes. Is AcidState typically 'fast enough'
that this is negligible?

* Speaking of global write locking - I'm also wondering about how you
should form your Query / Update events and how you should handle long
running Update events. For example, if I had code (using lenses) like:

updateSomething :: Update ...
updateSomething = do
something <- access (deep . inside)
(deep . inside) ~= someLongExpensiveOperation something

Is there a preferred way to deal with something like that? Should
these be split into Query and Update and them join them in IO?

Would someLongExpensiveOperation occur within the AcidState Core lock
or will it occur outside? If within, is there a preferred way to make
it occur outside?

How do I decide between splitting them and using them as a single
event?

* On a related note, are arguments passed in lazily going to be
calculated in the core lock instead of outside it? For example, if I
had: update as $ UpdateEvent (someLongExpensiveOperation something).

* Oh, and one more question - am I right to assume that Query events
do not block Update events and vice versa?

Thanks,
- Mike

[Jeremy Shaw]

Post by mifrai
* Should I use a single monolithic AcidState or should l use several
smaller ones? What are the advantages of one vs. the other?
I have a few guesses, but I'm not too familiar with AcidState - so
please correct me if I'm wrong. One advantage of a single monolithic
one that I can think of is that you can get transactions across every
piece of data in your code. A disadvantage is that all writes must
happen sequentially and thus you're essentially introducing a global
lock on your database writes. Is AcidState typically 'fast enough'
that this is negligible?

Correct. 'fast enough' is, of course, very context dependent. Without
knowing anything about your application, etc, I will say that AcidState is
fast enough.

I tend to use a fair number of independent acid-state components in my
applications. But that is just my taste. There are few possible advantages:

1. can lead to (subjectively) better code structure that is more modular
in nature

2. can be useful if you have optional components

3. is useful if you want to split things into reusable libraries

4. provides a primitive way to split the load across different servers.
Now that we have the remote API capability, you could split the acid-state
components up across multiple servers.

5. it is probably easier to take advantage of a multicore machine since
multiple updates/queries can be run in parallel.

But, I do not think there is a strong reason to always favor one over the
other.

Post by mifrai
* Speaking of global write locking - I'm also wondering about how you
should form your Query / Update events and how you should handle long
updateSomething :: Update ...
updateSomething = do
something <- access (deep . inside)
(deep . inside) ~= someLongExpensiveOperation something
Is there a preferred way to deal with something like that? Should
these be split into Query and Update and them join them in IO?
Would someLongExpensiveOperation occur within the AcidState Core lock
or will it occur outside? If within, is there a preferred way to make
it occur outside?
How do I decide between splitting them and using them as a single
event?

I guess you would first need to determine that someLongExpensiveOperation
really did take long enough that it is causing problems.

It will occur within the AcidState core lock.. because that is the only way
it can ensure that the operation is atomic.

If you wanted to split the operation up, then I think you would need to do
something like:

1. run a query operation that gets the value
2. perform someLongExpensiveOperation outside of a database transaction,
taking as long as needed
3. run an update that passes in the old value plus the new value
4. the update function then checks that value in the database has not
changed in the meantime, and updates it with the new value or fails
5. if the value changed then go back to #1.

Obviously, it is possible for that operation to never complete if something
else is aggressively updating the same entry. So, any solution has to be
based on the situation at hand.

Post by mifrai
* On a related note, are arguments passed in lazily going to be
calculated in the core lock instead of outside it? For example, if I
had: update as $ UpdateEvent (someLongExpensiveOperation something).

I think they are calculate in the core lock. Here is the relevant code with
an interesting comment:

scheduleLocalUpdate :: UpdateEvent event => LocalState (EventState event)
-> event -> IO (MVar (EventResult event))
scheduleLocalUpdate acidState event
= do mvar <- newEmptyMVar
let encoded = runPutLazy (safePut event)
--evaluate (Lazy.length encoded) -- It would be best to encode the
event before we lock the core
-- but it hurts performance /-:
modifyCoreState_ (localCore acidState) $ \st -> ...

Post by mifrai
* Oh, and one more question - am I right to assume that Query events
do not block Update events and vice versa?

I believe they do block each other. If you have code where you update a
value and then immediately query it, you should get back the new updated
value not some old value. And, you can not get back that new updated value
until it is certain that the log file has been written to disk.

However, I believe the locking period for Query events is very small. It
only locks core long enough to create a thunk that refers to the current
state. When you actually force the calculation.. that should happen outside
of the lock.

- jeremy

[David Himmelstrup]

Post by mifrai
* Should I use a single monolithic AcidState or should l use several
smaller ones? What are the advantages of one vs. the other?
I have a few guesses, but I'm not too familiar with AcidState - so
please correct me if I'm wrong. One advantage of a single monolithic
one that I can think of is that you can get transactions across every
piece of data in your code. A disadvantage is that all writes must
happen sequentially and thus you're essentially introducing a global
lock on your database writes. Is AcidState typically 'fast enough'
that this is negligible?
Correct. 'fast enough' is, of course, very context dependent. Without knowing anything about your application, etc, I will say that AcidState is fast enough.
1. can lead to (subjectively) better code structure that is more modular in nature
2. can be useful if you have optional components
3. is useful if you want to split things into reusable libraries
4. provides a primitive way to split the load across different servers. Now that we have the remote API capability, you could split the acid-state components up across multiple servers.
5. it is probably easier to take advantage of a multicore machine since multiple updates/queries can be run in parallel.
But, I do not think there is a strong reason to always favor one over the other.

I say put everything in one acid-state until you have a concrete reason to change it. It's easier to split an acid-state than to merge them.

Post by mifrai
* Speaking of global write locking - I'm also wondering about how you
should form your Query / Update events and how you should handle long
updateSomething :: Update ...
updateSomething = do
something <- access (deep . inside)
(deep . inside) ~= someLongExpensiveOperation something
Is there a preferred way to deal with something like that? Should
these be split into Query and Update and them join them in IO?
Would someLongExpensiveOperation occur within the AcidState Core lock
or will it occur outside? If within, is there a preferred way to make
it occur outside?
How do I decide between splitting them and using them as a single
event?
I guess you would first need to determine that someLongExpensiveOperation really did take long enough that it is causing problems.
It will occur within the AcidState core lock.. because that is the only way it can ensure that the operation is atomic.
1. run a query operation that gets the value
2. perform someLongExpensiveOperation outside of a database transaction, taking as long as needed
3. run an update that passes in the old value plus the new value
4. the update function then checks that value in the database has not changed in the meantime, and updates it with the new value or fails
5. if the value changed then go back to #1.
Obviously, it is possible for that operation to never complete if something else is aggressively updating the same entry. So, any solution has to be based on the situation at hand.
* On a related note, are arguments passed in lazily going to be
calculated in the core lock instead of outside it? For example, if I
had: update as $ UpdateEvent (someLongExpensiveOperation something).
I think they are calculate in the core lock.

They are for now but you can't rely on this. This will change at some point. Also it is obviously not true for backends such as Remote.

Post by mifrai
scheduleLocalUpdate :: UpdateEvent event => LocalState (EventState event) -> event -> IO (MVar (EventResult event))
scheduleLocalUpdate acidState event
= do mvar <- newEmptyMVar
let encoded = runPutLazy (safePut event)
--evaluate (Lazy.length encoded) -- It would be best to encode the event before we lock the core
modifyCoreState_ (localCore acidState) $ \st -> ...
* Oh, and one more question - am I right to assume that Query events
do not block Update events and vice versa?
I believe they do block each other. If you have code where you update a value and then immediately query it, you should get back the new updated value not some old value. And, you can not get back that new updated value until it is certain that the log file has been written to disk.
However, I believe the locking period for Query events is very small. It only locks core long enough to create a thunk that refers to the current state. When you actually force the calculation.. that should happen outside of the lock.

Queries don't block anything and are run in parallel with updates.

I say don't try to solve problem that don't exist. And definitely do not solve it by looking at the code. If you can't get the information you need from the documentation, push me to write better documentation. I will change the undocumented semantics of acid-state with no hesitation and I might even do it just out of spite. So to stay safe, use the documentation.

Cheers,
Lemmih.

[mifrai]

Thanks Lemmih. I wrote a reply earlier but I don't see it here. I don't
quite remember what I wrote but I'll try to reproduce it. I apologize if it
winds up being a duplicate (or duplicate'ish)

Post by David Himmelstrup
It's easier to split an acid-state than to merge them.

Do you have an example of how to do this? I can't think of a really good
way of doing it.

Post by David Himmelstrup
I say don't try to solve problem that don't exist

Thanks, that's good advice that I sometimes forget. I was just hoping to
make better decisions now instead of finding myself trapped with a bad one
later down the line. I also don't know how to get any metrics out of
acid-state, so out of likely misplaced fear, I was hoping to get a better
understanding of acid-state semantics and advice so I know I'd have at
least a glimmer of hope in understanding what's going on if I have
performance issues.

Post by David Himmelstrup

Post by mifrai
* Should I use a single monolithic AcidState or should l use several
smaller ones? What are the advantages of one vs. the other?
I have a few guesses, but I'm not too familiar with AcidState - so
please correct me if I'm wrong. One advantage of a single monolithic
one that I can think of is that you can get transactions across every
piece of data in your code. A disadvantage is that all writes must
happen sequentially and thus you're essentially introducing a global
lock on your database writes. Is AcidState typically 'fast enough'
that this is negligible?

Correct. 'fast enough' is, of course, very context dependent. Without
knowing anything about your application, etc, I will say that AcidState is
fast enough.
I tend to use a fair number of independent acid-state components in my
1. can lead to (subjectively) better code structure that is more modular in nature
2. can be useful if you have optional components
3. is useful if you want to split things into reusable libraries
4. provides a primitive way to split the load across different servers.
Now that we have the remote API capability, you could split the acid-state
components up across multiple servers.
5. it is probably easier to take advantage of a multicore machine since
multiple updates/queries can be run in parallel.
But, I do not think there is a strong reason to always favor one over the other.
I say put everything in one acid-state until you have a concrete reason to
change it. It's easier to split an acid-state than to merge them.

Post by mifrai
* Speaking of global write locking - I'm also wondering about how you
should form your Query / Update events and how you should handle long
updateSomething :: Update ...
updateSomething = do
something <- access (deep . inside)
(deep . inside) ~= someLongExpensiveOperation something
Is there a preferred way to deal with something like that? Should
these be split into Query and Update and them join them in IO?
Would someLongExpensiveOperation occur within the AcidState Core lock
or will it occur outside? If within, is there a preferred way to make
it occur outside?
How do I decide between splitting them and using them as a single
event?

I guess you would first need to determine that someLongExpensiveOperation
really did take long enough that it is causing problems.
It will occur within the AcidState core lock.. because that is the only
way it can ensure that the operation is atomic.
1. run a query operation that gets the value
2. perform someLongExpensiveOperation outside of a database transaction,
taking as long as needed
3. run an update that passes in the old value plus the new value
4. the update function then checks that value in the database has not
changed in the meantime, and updates it with the new value or fails
5. if the value changed then go back to #1.
Obviously, it is possible for that operation to never complete if
something else is aggressively updating the same entry. So, any solution
has to be based on the situation at hand.

Post by mifrai
* On a related note, are arguments passed in lazily going to be
calculated in the core lock instead of outside it? For example, if I
had: update as $ UpdateEvent (someLongExpensiveOperation something).

I think they are calculate in the core lock.
They are for now but you can't rely on this. This will change at some
point. Also it is obviously not true for backends such as Remote.
scheduleLocalUpdate :: UpdateEvent event => LocalState (EventState event)
-> event -> IO (MVar (EventResult event))
scheduleLocalUpdate acidState event
= do mvar <- newEmptyMVar
let encoded = runPutLazy (safePut event)
--evaluate (Lazy.length encoded) -- It would be best to encode
the event before we lock the core
modifyCoreState_ (localCore acidState) $ \st -> ...

Post by mifrai
* Oh, and one more question - am I right to assume that Query events
do not block Update events and vice versa?

I believe they do block each other. If you have code where you update a
value and then immediately query it, you should get back the new updated
value not some old value. And, you can not get back that new updated value
until it is certain that the log file has been written to disk.
However, I believe the locking period for Query events is very small. It
only locks core long enough to create a thunk that refers to the current
state. When you actually force the calculation.. that should happen outside
of the lock.
Queries don't block anything and are run in parallel with updates.
I say don't try to solve problem that don't exist. And definitely do not
solve it by looking at the code. If you can't get the information you need
from the documentation, push me to write better documentation. I will
change the undocumented semantics of acid-state with no hesitation and I
might even do it just out of spite. So to stay safe, use the documentation.
Cheers,
Lemmih.

[Jeremy Shaw]

Post by David Himmelstrup
I say put everything in one acid-state until you have a concrete reason to
change it. It's easier to split an acid-state than to merge them.

Yeah, I would agree with this. I would say, definitely do *not* split
things into to multiple AcidState components solely because you think it
could somehow provide better performance someday. The split should
definitely be motivated by something concrete. I would also suggest that
you shouldn't feel like you must keep everything in one AcidState if you
feel like your application requires multiple AcidStates (due to optional
components, dividing coded into multiple, reusable libraries, etc).

I say don't try to solve problem that don't exist. And definitely do not

Post by David Himmelstrup
solve it by looking at the code. If you can't get the information you need
from the documentation, push me to write better documentation. I will
change the undocumented semantics of acid-state with no hesitation and I
might even do it just out of spite. So to stay safe, use the documentation.

Will do.

- jeremy

[mifrai]

Wow, thanks for the great reply. It was very helpful.

If you don't mind, I do have some followup questions about managing
multiple AcidStates though.

When using multiple AcidStates, is there a way to get transactions
between them? Or do you have to design carefully to make sure all
transactions happen in one AcidState?

If you find you've made a mistake, what is the recommended way of
moving data from one AcidState to another? From what I understand,
SafeCopy is only going to help when the data is sitting in one data
structure.

Is there a way to get AcidState to provide some sort of metrics on
production machines that would allow one to find performance issues?
Ie. is it possible to find out if there is a bottleneck to your system
given production traffic and, if there is, which Update/Query Event?

- Mike

Post by Jeremy Shaw

Post by mifrai
* Should I use a single monolithic AcidState or should l use several
smaller ones? What are the advantages of one vs. the other?
I have a few guesses, but I'm not too familiar with AcidState - so
please correct me if I'm wrong. One advantage of a single monolithic
one that I can think of is that you can get transactions across every
piece of data in your code. A disadvantage is that all writes must
happen sequentially and thus you're essentially introducing a global
lock on your database writes. Is AcidState typically 'fast enough'
that this is negligible?

Correct. 'fast enough' is, of course, very context dependent. Without
knowing anything about your application, etc, I will say that AcidState is
fast enough.
I tend to use a fair number of independent acid-state components in my
 1. can lead to (subjectively) better code structure that is more modular
in nature
 2. can be useful if you have optional components
 3. is useful if you want to split things into reusable libraries
 4. provides a primitive way to split the load across different servers.
Now that we have the remote API capability, you could split the acid-state
components up across multiple servers.
 5. it is probably easier to take advantage of a multicore machine since
multiple updates/queries can be run in parallel.
But, I do not think there is a strong reason to always favor one over the
other.

Post by mifrai
* Speaking of global write locking - I'm also wondering about how you
should form your Query / Update events and how you should handle long
 updateSomething :: Update ...
 updateSomething = do
   something <- access (deep . inside)
   (deep . inside) ~= someLongExpensiveOperation something
Is there a preferred way to deal with something like that? Should
these be split into Query and Update and them join them in IO?
Would someLongExpensiveOperation occur within the AcidState Core lock
or will it occur outside? If within, is there a preferred way to make
it occur outside?
How do I decide between splitting them and using them as a single
event?

I guess you would first need to determine that someLongExpensiveOperation
really did take long enough that it is causing problems.
It will occur within the AcidState core lock.. because that is the only way
it can ensure that the operation is atomic.
If you wanted to split the operation up, then I think you would need to do
 1. run a query operation that gets the value
 2. perform someLongExpensiveOperation outside of a database transaction,
taking as long as needed
 3. run an update that passes in the old value plus the new value
 4. the update function then checks that value in the database has not
changed in the meantime, and updates it with the new value or fails
 5. if the value changed then go back to #1.
Obviously, it is possible for that operation to never complete if something
else is aggressively updating the same entry. So, any solution has to be
based on the situation at hand.

Post by mifrai
* On a related note, are arguments passed in lazily going to be
calculated in the core lock instead of outside it? For example, if I
had: update as $ UpdateEvent (someLongExpensiveOperation something).

I think they are calculate in the core lock. Here is the relevant code with
scheduleLocalUpdate :: UpdateEvent event => LocalState (EventState event)
-> event -> IO (MVar (EventResult event))
scheduleLocalUpdate acidState event
    = do mvar <- newEmptyMVar
         let encoded = runPutLazy (safePut event)
         --evaluate (Lazy.length encoded) -- It would be best to encode the
event before we lock the core
         modifyCoreState_ (localCore acidState) $ \st -> ...

Post by mifrai
* Oh, and one more question - am I right to assume that Query events
do not block Update events and vice versa?

I believe they do block each other. If you have code where you update a
value and then immediately query it, you should get back the new updated
value not some old value. And, you can not get back that new updated value
until it is certain that the log file has been written to disk.
However, I believe the locking period for Query events is very small. It
only locks core long enough to create a thunk that refers to the current
state. When you actually force the calculation.. that should happen outside
of the lock.
- jeremy

[Jeremy Shaw]

Post by mifrai
When using multiple AcidStates, is there a way to get transactions
between them? Or do you have to design carefully to make sure all
transactions happen in one AcidState?

There is no way to get transactions across multiple acid-states. If you
need system wide transactions then it is best to put everything in a single
AcidState. It would certainly be possible to extend acid-state so that you
could do a transaction across multiple AcidStates .. but it would require a
lot of synchronization overhead that would likely kill performance. And
would make the code more complicated. And there is usually some way to work
around the lack of a multi-AcidState transaction. So, there is not a lot of
incentive to add that to acid-state.

If you want to use multiple AcidStates, then you need to design your system
so that things which must be a single transaction are confined to a single
AcidState. You also need to be a bit flexible on how transactional you
require things to be.

In my experience, a majority of queries and updates are confined to a
single AcidState component. And, in the cases where you have to do multiple
queries or updates, it is usually not hard to get it 'right'.

For example, in many cases where you are doing multiple queries, it is ok
if some information changes in between the two queries. For example, let's
say you store Posts in one AcidState and the mapping between UserId and
Username in another table. When attempting to render a Post, you first
fetch the post which contains the poster's UserId, and then you look up the
Username associated with that UserId in a second query. Now what happens if
the user who made the post changes their UserName between the two queries?
All that will happen is the post will be displayed with the updated
UserName instead of the old username. One could very easily argue that that
is actually a better outcome.

The trickiest situation is typically when you have to do multiple updates.
For example, let's say you are creating a new user in the system. If you
store the UserId and authentication information in one AcidState and the
ProfileData (like their UserName) in another AcidState, then you could end
up in the situation where the UserId got added, but the server died before
the ProfileData was added. In this case, I would have the login code expect
that the ProfileData could be missing, and so it would check for that and
prompt you to enter it if it got lost the first time.

That is how happstack-authenticate works. In fact, when creating a new user
account, we do not even bother to collect the ProfileData right away.
Instead, we create a new user account with missing ProfileData and log the
user in. Then the code that detects missing ProfileData kicks in. So, it's
not really even any extra code. (This design also allows the definition and
handling of ProfileData to happen entirely in the end-user's code. That way
they are not forced to use some limited, generalized ProfileData type that
is dictated by happstack-authenticate).

Clearly using multiple AcidState components requires more thinking.
Additionally, there are no guarantees that using multiple AcidState
components is going to provide you with better performance. It could even
hurt performance. So, if your application is easy to write as a single
AcidState component, you should definitely feel free to do that. If
performance ever becomes a problem, then you can actually measure the
problem and apply the right solution to it at that time.

In a system like Hackage 2, using multiple AcidStates makes sense, because
Hackage 2 is designed to support a bunch of optional components that can
have their own local state. If those components are not loaded (or even
compiled into the server), then there is no reason to load the state
associated with them. So, it works well for the design architecture.

If you find you've made a mistake, what is the recommended way of

Post by mifrai
moving data from one AcidState to another? From what I understand,
SafeCopy is only going to help when the data is sitting in one data
structure.

Generally, you would just write a little code that loads up both AcidState
components and manually copies the data from one component to another. If
your code is just for a single site, such that this operation only needs to
be done once, then it is pretty trivial to just make a little one-off
Main.hs that imports the related types and does the migration. Then you run
it once and you are done.

If you are developing something like gitit, that you intend to be used by
multiple clients, then you would do approximately the same thing, but
instead of writing a standalone app, you would need to integrate it into
your main app, and then add some detection code to your app that would call
the special migration path when required.

Post by mifrai
Is there a way to get AcidState to provide some sort of metrics on
production machines that would allow one to find performance issues?

Not outside of the usual profiling tools. One of the things I would like to
focus on for Happstack 8 is adding instrumentation to happstack-server and
acid-state so that you can get feedback.

It should be possible to add some pretty decent instrumentation to
acid-state already with out modifying the core library at all. All calls go
through the query/update functions. It would be easy to make wrappers
around query/update which recorded the events that were being called and
timed how long they take to run.

Creating instrumentation that produces *useful* and accurate information
is, of course, tricky. But, it would certainly be a great addition. I have
added it to the acid-state wishlist,

http://code.google.com/p/happstack/wiki/AcidStateWishlist?ts=1332071611&updated=AcidStateWishlist

Truthfully, the biggest issue people have with acid-state right now
performance wise is figuring out which of their datastructure's is using an
unnecessarily large amount of RAM. update/query times are usually pretty
darn fast. And, you can use the existing GHC profiler to debug that.

Hope this helps!
- jeremy

[Petter Bergman]

Post by mifrai
* Speaking of global write locking - I'm also wondering about how you
should form your Query / Update events and how you should handle long
running Update events.

The solution to this will have to depend on the context as others have
pointed out. If the long operation is an "administrator"-type operation
that seldom gets run you could just let it block and then provide some
feedback to other users like "system is undergoing temporary
maintenance/update/something".