Standing up for the Aborigines resulted in James rbeing regarded as a failed colonist, a lonely figure who attracted derision as he stood against the weight of colonial opinion on Aboriginal rights. After just seven months on the Goulburn he tendered his resignation under controversial circumstances which highlight the rift between his humanitarian views and the Port Phillip of 1840.
In June 1840 James Dredge returned to Melbourne to settle back into his beloved Methodist community. He returned to preaching but maintained a strong desire to help the Aboriginal people. He kept close contact with the Wesley Mission and by 1842 had begun to appreciate the importance of the land not just as a source of food for Aboriginal people but as a place of immense cultural significance.
Dredge was appointed head teacher of a new Methodist school and opened a small chinaware shop in the front room of his Collins Street house in August 1840. Financial problems still plagued him, however, and he was advised to declare himself bankrupt in October 1841. He died in 1846 on board the Arab, two days before reaching Land's End. Where Robinson acquired wealth and prestige through his work with the Aborigines, returning in comfort to Bath, Dredge lost both his health and his livelihood.