I would be really hesitant to use a plastic container except temporarily while you're cleaning the tank. Will you be able to attach a power filter on the side? What will happen when the thermometer is sitting next to the plastic? I assume you're meaning clear hard plastic so you can see the fish? If so - these crack extremely easily.
If you are thinking of using the transluscent white aquarium (ie., Rubbermaid) - you won't see the fish without looking down into the container - which isn't much of a view. The plastic is much softer and possibly would melt when a heater comes on.
With the plastic containers they use for the bettas (5gal or smaller) they generally use a thicker plastic but there are still complaints about cracking. You can purchase an Acrylic aquarium for about 2x the cost of a glass one. Finally you don't want a tight fitting lid on this - how will the tank receive oxygen - you'd need a number of air pumps.
In the end you'd probably save about $25 over a regular/no frills glass 10 gallon aquarium and you would have something DIY that I wouldn't put in my house. But some people really get into that DIY look. And if it's in your basement or something nobody would care (cold dampness and mold in a basement though), You could try drilling holes in the top lid for air stones, thermometers and use a filter that's either 100% outside the tank or 100% inside the tank (a canister model which can be more expensive than one that goes over the back - I've never used one myself).
Also look on ebay market place - in my area you can get about a dozen or so aquariums for less than $100. Some come with everything and are 55 gallons or so. The issue? You have to haul it yourself and most people can't do that.
Don't cheap out when it comes to aquariums. You can get a glass fishbowl for $10 - and your fish would live a month or so with daily water changes but would eventually die due to lack of O2 in the water, stress of a view all around them and the small size.
At pet's mart for $75.99 you can get a glass aquarium with a filter, lid with LED lights, and a thermometer. I don't think you could piece together what you are thinking about for much less. Heck the lights $30, Filter $15-30, Thermometer: $10, Clear Plastic container (about 5 gallons??) : $12-20. Then you still need a stand.