How is it that in Australia, one of the richest countries in the world, we have a housing crisis where hundreds of thousands of renters can’t afford a roof over their head?

To figure out why rents are soaring, we need to look at the broader political disease: we have spent about two decades trying to screw up the housing market and we have achieved catastrophic success.

The Australian housing market is broken. For too many low- and middle-income earners the Great Australian Dream of owning a home is now just a fantasy.

As house prices have surged, outpacing incomes, huge numbers of people who want to buy a home but can’t are forced into the rental market, pushing up rents too.

The best long-term solution to the structural problems of the rental crisis is to build more public housing.

Building more public housing doesn’t just help those in the most financial need. It helps everyone. If Australia had more public housing, then there would be less people trying to rent in the private market, in turn freeing up properties for other renters. Public housing also increases the supply of housing putting downward pressure on house prices, making it easier for those trying to buy their own home.

Traditionally, it is state governments who build public housing. This has led to buck passing, with successive federal governments saying it is a state government responsibility and state governments saying they simply don’t have the money to do it.

The boom in public housing in the post war period was built by state governments but largely funded by the federal government. That’s because the federal government has all the money. If we want more public housing it will need to happen that way again.

The next problem is the explosion in the wrong type of demand.

After changes to the capital gains tax discount in 1999, investors have taken an ever-increasing stake in residential property. The size of the net rental loss (the amount that is claimed as negative gearing) has also boomed.

This explosion in investment demand has pushed out owner occupiers and forced more people to rent. Australia has moved from housing being shelter to housing being an investment.

There are not two separate markets for rental properties and owner-occupied properties. When a house goes up for sale, whether it becomes an owner-occupied house, or a rental property depends on who buys it.

Let’s assume that the government introduces policy changes that make it harder to make money from investing in housing. The various property and investment groups shout that this will only hurt renters because there will be fewer rental properties which will further force up rents and make even more people homeless. But this is nonsense.

When an investor gets out of housing, the house doesn’t cease to exist. Instead, it gets sold. If things have changed to discourage investors, then who are they selling to. Those renters who really want to buy their own place. So, the number of rental properties goes down, but the number of renters also goes down, as some renters buy their own place and stop being renters.

That’s why the second solution to fixing the rental crisis could be to reform incentives that give an advantage to investors over owner occupiers.

Reforms to negative gearing and the capital gains tax discount will over the longer term see house prices lower than they otherwise would be and since the price of housing is one of the main determinants in how much rent is, it will also decrease rents.

These two reforms: more public housing and reform of investor tax concessions, are not going to be easy. They are also longer-term reforms. Reforms that help renters right now is also needed.

In immediate terms, the third reform is to renters’ rights: that means we need a ban on “no cause” evictions, where landlords can kick tenants out for no reason.

We have gone from a system in Australia where the expectation was that most people would only rent temporarily and would eventually go on to own their own home. This has meant that renters have relatively few rights. This needs to change.

Not all countries have a housing system that assumes that most people will own their own home. In these countries renters have more rights.

The Damocles sword of “no cause” evictions hangs over the heads of many Australian renters, undermining a range of other “rights” they may have in theory, but will not or can not access in reality for fear of eviction. These rights include protection from unlawful discrimination, and unreasonable rental increases.

The final thing we need to do is make renting more affordable by either increasing rent assistance, or looking at rental caps, similar to many countries in Europe and some US states. Years of neoliberalism have trained us to believe that we don’t have the ability to legislate price controls for housing – but in a housing crisis, we are entitled as a democracy to decide if housing is more important than landlords collecting exorbitant rents. This should be a democratic question rather than an economic one.

Decades of underinvestment in public housing and huge tax concessions to encourage investment by landlords has wreaked havoc on our rental market. We need to make big changes to reverse the crisis that we have created.

Matt Grudnoff is senior economist and director of the Australia Institute’s economics program. Twitter: @MattGrudnoff