Joe Rattz to the Rescue

In my previous post, I showed a simple way to create animations in WPF using a sequence of images. That demo app used a LINQ query to get a sequence of BitmapImage objects based on the PNG files in a certain folder. Joe Rattz, the LINQ guru, dropped a comment on that post and also included his e-mail address. Big mistake!! πŸ˜€

I sent him an e-mail, asking if the BitmapImage objects in that LINQ query are created every time a new enumerator is created and iterated over. When I send an expert an e-mail I never expect to hear back. I certainly don’t reply to all of the questions that people send me! There are only 24 hours in a day. πŸ˜›

Luckily, Joe replied. Thanks Joe! It turns out that the answer is yes, the LINQ query is creating new BitmapImage objects for each new enumerator, as the enumerator is iterated. This is not a good thing. Since my demo app repeats the animation indefinitely, a potentially huge number of unnecessary BitmapImage objects were being created by the LINQ query.

The reason for all this is the fact that my LINQ query was only using deferred query operators. LINQ has deferred and non-deferred operators, as Joe’s book explains. If you use all deferred operators, the query’s results are not cached. Every time you iterate the resultset the query creates new objects for you. In some cases that is what you want. In my case, it is undesirable.

Working around this turned out to be easy, especially since Joe e-mailed me the answer.πŸ˜‰

I needed to apply a non-deferred operator to the query. This would cause the results of the query to be cached, so that all enumerators created against the query iterate the same set of objects. Here is the new code-behind for the demo app:

Notice how the entire LINQ query is wrapped in parenthesis and then a call to the ToList extension method is placed at the end. ToList is a non-deferred query operator, which is the magic that fixes the issue.

If you want the source code for this app, download it here. Be sure to change the file extension from .DOC to .ZIP and then decompress the file.

13 Responses to Joe Rattz to the Rescue

  1. Wow! Great to know info, especially for LINQ newbies like myself!

    I loved the simplicity of it so much I had to try it out. Heres a link to the same code but with a particle illusion png sequence =D: http://www.youtube.com/watch?v=94aj65FXwTo

    -Jer

  2. Josh Smith says:

    Oh yea! Now that is a compelling use of this image-based animation trick. What a great idea…use it as a background! I love it. nice job

  3. So when you don’t use an explicit operator like ToList() and you simply return the IQueryable (correct?), what operator is used upon execution of the query and why is it not cached?

  4. Josh Smith says:

    Ryan,

    I don’t think you are correct about the IQueryable of T being returned w/o an explicit operator. It is an IEnumerable of T instead. IIRC, Joe’s book stated that IQueryable of T is returned for LINQ to SQL queries, but not LINQ to Objects queries.

    I’m not sure what you mean when you ask what operator is used upon execution of the query. All of the operators in your query are used. If they are all deferred operators, then the resultset is not cached because each element in the output sequence is created on demand. Does that answer your question?

    Josh

  5. I guess I’m curious as to what is special about the result IEnumerable.ToList versus the original IEnumerable that causes the caching. That is, when you call _images.GetEnumerable() on the result of the ToList() why does that IEnumerator use the same BitmapImage instances as another IEnumerable instance? Are you saying that the ToListed GetEnumerator does some sort of shallow copy?

  6. Josh Smith says:

    Ryan,

    I’m by no means a LINQ expert, but here’s my 2c. Calling the ToList extension method on an IEnumerable of T causes the entire sequence to be iterated and stored in a List of T. So yeah, what you said…a shallow copy is created. The enumerator created by calling GetEnumerator() on the IEnumerable of T created by a LINQ query *yields* the elements in the sequence. The important distinction is that those enumerators create the sequence as you iterate over it, instead of dumping the whole thing into memory at once.

    Josh

  7. Joe Rattz says:

    Ryan and Josh, the issue isn’t that there is a shallow copy taking place. The issue is that without calling the ToList operator, every time Josh enumerates the sequence, the query is exectuted and therefore BitmapImage objects get created. By calling the ToList operator, a List is created once, immediately when the _images variable is initialized. After that, he is enumerating the same List over and over again.

  8. Josh Smith says:

    Thanks Joe. That makes sense to me.

  9. Judah says:

    Yep, Joe’s got i:, you’ve got to call ToList, ToArray, or some other non-deferred operator if you don’t want your query to be re-evaluated each time you iterate over it with foreach.

    Usually this isn’t a problem: you generate a query, you iterate over it once, and you’re done. Even if you’ve got a case where you iterate multiple times, it’s usually OK.

    The problem here for Josh was that his query was allocating a new BitmapImage for each iteration. Because his query was evaluated over and over again ad infinitum, this was wasting resources. It was also retrieving all the .png files each iteration as well. Each time his query was evaluated, it was allocating new objects and touching the hard drive. Since Josh’s animation repeats infinitely, the query gets evaluated time and again, constantly touching the hard drive and allocating new bitmaps.

    By forcing the evaluation of the list once up front (the ToList() call), future iterations are merely iterations over a List — no hard drive touching, no new bitmap allocations.

  10. Interesting, I would not have guessed that the resulting IEnumerator instance returned from GetEnumerator of whatever IEnumerable implementation is returned from the non-ToList select (Dr. Suess anyone?) would execute the query again.

    So, if anyone knows off-hand, does GetEnumerator internally call ToList or ToArray and return that instance’s IEnumerator?

  11. Josh Smith says:

    Ryan,

    You have a masterful command of technobabble!! πŸ˜€

    I think the enumerator returned by the “LINQ IEnumerable implementation” uses something like the C# ‘yield’ operator to create a routine that returns the correct object when the enumerator’s Current property is accessed. that’s my understanding of it, at least.

    Josh

  12. sacha says:

    Yeah Joe Rattz is cool, I am a big fan of his book (thanks for the book Joe). LINQ is cool, but you do have to watch out for stuff like this, and Joes book really helps, its great.

    Good post Josh, good book Joe, well done both of you.

  13. Finished reading Joe’s book the other day. (Thanks Joe!) Now I’ve gone all Linq and Lambda happy! FYI Josh, you can get around the deferred query limitation by checking out the Continuous Linq project on CodePlex written by Kevin Hoffman which is essentially a set of extension methods for Linq to provide for dynamically updating Linq views: http://dotnetaddict.dotnetdevelopersjournal.com/clinq_release11.htm

    Regards,

    Roland

%d bloggers like this: