This paper studies on several advanced Autoencoders like Sparse Autoencoder, Denoising Autoencoder, and Tiedweight strategy for Autoencoder; explores on the principal of improving the feature representation with these Autoencoders, which have been certificated in the handwritten numeral recognition. The performances of feature representation with these Autoencoders are compared by setting the Autoencoders parameters. Experimental results show that the Sparse and Denoising strategies have great improvment to the performance of Autoencoders.