I watched this video last night: Reverse engineering techniques to find security bugs: A case study of the ANI exploit

http://video.google.com/videoplay?docid=-7185841369679533904

From the blurb:

Alex Sotirov is a vulnerability engineer at determina. He will discuss some latest techniques in reverse engineering software to find vulnerabilities. Particularly, he'll discuss his technique that lead him to find the ANI bug (a critical new bug in WinXP and Vista).

Alex will describe the tools he uses for reverse engineering and show how he reverse engineered ANI Bug. He will continue to discussed Windows security mechanisms (ASLR, /GS) and describe how ANI exploit bypasses them.

---

Alex is one of the good guys. He works at determina on their intrusion prevention system, his job is to create exploits so that they can test if their software can detect and catch them. He often creates exploits by looking at the patches Microsoft releases to find out what bugs they fix.

In the presentation, Alex shows how to use a disassembly tool to analyse the differences between two dlls -- the original and the patched ones. He also explains some of the exploit-protection mechanisms, and how exploit authors can get around them:

• /GS compiler switch which inserts a trap to check if the return pointer has been overridden (only works for functions with arrays in them, as an optimisation)
• Data Execution Protection (DEP), which is a CPU feature to disable execution of code within data segments of memory. Stack overflows write into data blocks, and are thwarted when trying to jump to those blocks. It's only opt-in on desktop Windows, even on Vista (due to compat reasons).
• Address space layout randomization (ASLR) which re-arranges the executable in memory so that jump locations are hard to pre-determine (but there are only 256 possible places, so it's still possible to guess the location).
  

Hints on how to design secure software:

• Pick a good language and platform (e.g. Java, Python and I assume .NET). This avoids common pitfalls in languages like C++ and PHP.
  
• Design your app to isolate components along trust boundaries. Develop a formal specification that details how areas of code that are accessed by users of different privilege interact with each other.
  

Things to avoid:

• ActiveX. Always avoid.
• Google Desktop Search web integration. [Must've been a popular bullet point in the room!] Exposes all local desktop search data to the possibility of a cross domain vulnerability in a browser. Securing the product relies on a browser being secure -- not a good idea.
• Really hard to add security to an existing codebase (e.g. Windows, Oracle DB)
  

Take outs:

• Assume software has security bugs. Build in things that will make exploitation harder, and will minimise damage.
• Avoid single sign-on for web services, as if one app is exploited, other services can be attacked.
  

Alex ended with a diagram showing the exploit prevention features in different OS's:



OSX doesn't use many of the current protection techniques that other OS's currently do. I guess this shows how Microsoft are trying harder with each release, but Apple are finding it hard.