Shaco is one of the funniest champions in League of Legends. I've tried DPS and AP builds, but I found hybrid Shaco works better in most cases.
Shaco's abilities scale with AP, so the power of the AP combined with AS works pretty nice.
I usually play 3v3 and this build gets my win. You can try it at 5v5, but I don't know if it's the best way to build Shaco for 5v5.
Enjoy! ;)

Pros
- A lot of damage mid-late game.
- Natural escape talent (Deceive).
- Nice harasser.
- Thornmail doesn't hurt hybrid Shaco that much.

Cons
- Slow farming.
- Squishy.
- Very weak until you get Guinsoo's Rageblade.

Magic penetration Marks (Mark of Insight) are very effective with every AP champ. You can use Mana regen or AP Seals; I use Dodge Seals for every champ. Cooldown reduction Glyphs work well on almost every champ in LoL too, but you can try AP Glyphs if you want. You can use Quintessence of Potency (AP) or Quintessence of Insight (Mag pen).

Your main damage will be magical, so you should focus on AP, mag dmg and mana masteries. A point for exhaust and ignite helps.

Two-Shiv Poison (E) will be your main harassment tool and also your main source of damage, so it's the first skill to master.
1 early point for Deceive (Q) so that you can "flash" in and out of fights and stealth.
Jack in the Box (W) does some nice damage late game, but it's most important feature is the AoE fear. You should spam JitB as wards as much as you can. It can help you stop an enemies from fleeing or from killing you if placed right.
Hallucinate (R) creates a clone that works great for destroying turrets faster (the turret will target the clone first unless you dmg an enemy champ), for deceiving enemies to attack your clone and give you some time to escape, or for some extra AoE dmg (when your clone is killed it does magic dmg to surrounding enemies).

Exhaust can help you escape, pursue, take less dmg and do more dmg in 1 on 1 fights. When the enemy is about to flee with very few HP, Ignite is the spell you would want to use to kill him (can kill an "ulted" Trynd if you use it right on time).
You can use Flash too. In combination with Deceive you can travel some distance and through walls in and out of a fight.

The item build that I provide at the top of the page is the one I use most. However, you should always keep an eye on your enemies' items and what champs are they using.
You can start with Doran's Ring for some extra AP, health and mana regen, but Shaco is better played by harassing and hiding until the enemy is low on HP so that you can deal the killing blow, so boots give you nice mobility. If your enemies have a lot of CC you can buy Merc's Boots. You can also get Sorc's boots if you prefer some extra dmg instead of mobility.
Guinsoo's Rageblade is one of the greatest hybrid build item. You should always try to keep it's stacks at 8, so that you get the maximum AS and AP possible.
For Guinsoo, Malady and Nassor's Tooth always get the AP part 1st, because, as I said before, you main dmg will be magic.
The "early" Vampiric Scepter will keep your HP good while farming. It will become the Bilgewater Cutlass part for your Hextech Gunblade later.
Hextech Gunblade's life steal and spell vamp can help you a lot more than you think in a fight (maybe except against Jax or Olaf), but you can try Bloodthirster instead.
You can get Rabaddon's Deathcap before if you want to (just not before Guinsoo's Rageblade).
I haven't tried Lich Bane yet, but it might help. If you try it, tell me how well does it work please ;)

Shaco isn't just an assassin. He can help his friends too. You can put a JitB between your enemy and an ally to fear him apart. You can use Two-Shiv Poison to slow enemies so that you and your teammates can kill him or to prevent an enemy from reaching an ally.
The only thing to remember here is that you're squishy, so be careful.

Hybrid Shaco does quite a lot of damage, but he's not immortal, so you must play carefully. Harassing with you "E" from behind the minions until the enemy has very low HP is a very good way to win. Spam "W" every time you can (you can put one outside dragon when you're killing it, so that you have 2 more secs to escape if the enemy comes).
The items listed above are usually a good choice, but you should always keep an eye on what is the enemy building.
Any suggestions please comment :)
Don't forget to rate my guide please.
Have fun and get some wins ;)