Hi Piyush,
It sounds like you are on an older version too since all of the code you mention has since been rewritten. For instance we now have new code that does the window chrome rendering (OnSourceInitialized) and theming is different in general. Although for themes, we do define hundreds of brushes so perhaps that will take some time to load regardless.
Arthur, we aren't aware of any memory leaks at this time. I know that we worked with you in the past on a couple memory issues and resolved them. And remember, you need to use a real memory profiler (like from memprofiler.com) to detect leaks. You can't trust task manager.