Simply told, Avant-Garde is music with new and inventive twists to traditional metal arrangements.  Obscure instruments and theatrical elements are employed  to create a new level of artistic musicianship.
Avant-Garde is often not considered a seperate genre, rather a title that can be given to a band of any genre.
For example, we have Therion.  Once a straight-up death metal band, they have slowly morphed into a definite Avant-Garde band, with multiple opera style vocalists entirely replacing the usual Death metal growl.  Listen to their excellent album 'Secret of the Runes' for a unique experience.
Avant-Garde bands are often compared to Progressive metal bands, but there is a distinct difference, that Prog bands perform intricate and technical arrangements, while Avant-Garde need not be very difficult, only innovative.
THERION
The first Avant-Garde band I ever heard.  And still one of my favorites.
They started off as pureDeath metal, but became more and more symphonic until they could no longer be classified as Death metal. 
I've heard bits and pieces from all of their albums, but only have three complete albums in my possession.  The new
Lemuria/ Sirius B, Secret of the Runes , Theli and VovinSecret is the most Avant-garde of their albums and also the least heavy, but it is probably my favorite.  It completely dispenses with death metal growls, which are replaced by operatic voices and choirs!   Excellent stuff!

ARCTURUS
One of my favorite bands now,  ARCTURUS is a side-band for several members of prominent black metal bands, including the legendary drummer known as Hellhammer, formerly of MAYHEM.  This band utilizes a lot of piano and keyboards as well as short sections of techno.
When they started, ARCTURUS used traditional black metal screaming vocals, but they have since changed to clean, haunting vocals (which I like more). 
Also I must point out that ARCTURUS write beautiful lyrics!
ALL their albums are must haves, I really cannot say which one is the best.

GREEN CARNATION
Ok, the debut album consists of one song.  One hour-long song, that is.  And it succeeds because it never gets boring.  Using tons of guest musicians and sound samples, listening to this cd is like watching a movie.  It would be classified as doom or stoner metal because of the usually slow tempo.  The vocals are, for the most part, clean and monotone which gives the music a sort of dream-like feel which is further enhanced by the repetition of guitar riffs.
By the way, the album is called
Light of Day, Day of Darkness.

SOLEFALD
Here's a really weird band.  Black metal influences are present but with their newest release there is much progression. 
In Harmonia Universali is this great album.  Featuring growled and clear angelic vocals and a saxaphone!  I recommend this album if you are not but off by seemingly nonsensical lyrics. Their previous release, Pills Against the Ageless Ills is no where near as good and is rather boring.
Add your text here