Bringing the railways back into public ownership was a key plank of Jeremy Corbyn's successful leadership bid.
Transport spokeswoman Lilian Greenwood said the railway was inefficient and among the most expensive in Europe.
She also called for the new HS2 high-speed rail link from London to the Midlands to become a "public service under public ownership".
Members overwhelmingly backed a policy statement drawn up by the National Executive Committee, calling for existing franchises to be nationalised when they come to an end and for a new "public operator" to reinvest profits by private rail operators into cutting fares and rail infrastructure.
Ms Greenwood said a far-reaching shake-up was needed to deal with the "fragmented" network and a "broken" franchise system.
"Twenty years ago we were told that privatisation would deliver cheaper tickets and lower subsidies," she said. "What we have been left with is some of the most expensive tickets in Europe and an efficiency gap of 40%.
"The Tories will tell you the railways cannot change... it falls to us to set out a better way. That's why I say it is time for our railways to be run under public ownership, in the public interest, with affordable fares for all."
Five existing franchises are due to expire between 2020 and 2025. Labour has insisted nationalising them will not involve any costs as the state will only step in once their contracts for delivering services had run out.
More investment, she said, was needed in inter-city services, greater devolution of rail services to the English regions and a high-speed rail link connecting London, the Midlands and the North of England that was "an integrated national asset that our country can be proud of".
Also speaking at the conference, the head of the TSSA rail union said he was "cock-a-hoop" that Labour - which at May's election backed creating a public rail company to compete with private firms - was now going further and putting "red water" between itself and the Conservatives.
"We will be running the railways in the interests of passengers and the taxpayers not greedy shareholders," Manuel Cortes said. "Our party is united behind the policy."
And Tosh McDonald, from Aslef, said the perverse situation facing the UK's railways was summed up by the fact that the three preferred bidders for the next Northern Rail franchise were state-run rail firms from Germany, the Netherlands and France.
"I don't blame them for that. The Germans boast to our members that their interest in Britain's railways subsidises the German taxpayer. It is a no-brainer. It should be run by the state, for the people and by the people and any profits that are made should go to helping with housing and the NHS."
The government has insisted the railway has been a success story over the past decade and it is responding to booming passenger numbers by overseeing the largest investment programme in the network for 100 years.
Separately, Labour has commissioned an independent study into why home ownership has fallen. Shadow housing minister John Healey said Peter Redfern, chief executive of house builder Taylor Wimpey, would report his findings by next summer.