Magnus submonoids and membership problems in one-relator, surface and hyperbolic groups

Motivated by its applications to the word problem for one-relator inverse monoids, via results of Ivanov, Margolis, and Meakin (2001), we prove several decidability and undecidability results about the submonoid membership problem in one-relator, surface and hyperbolic groups. We prove a general result that gives sufficient conditions under which membership in submonoids of certain free-by-cyclic one-relator groups is decidable. We apply this result to show that membership in various families of submonoids of surface groups is decidable. We study graded submonoids of one-relator groups, proving results that give conditions under which such a submonoid has decidable membership with linear distortion. These results are then applied to improve on a result of Margolis, Meakin and Šunik by showing the prefix monoid of a surface group has linear distortion. Our results significantly extend previously known positive results proving decidability of membership in (non group) submonoids of surface groups. We also apply our general results, and other techniques, to show that the Magnus submonoid membership problem is decidable in several families of one-relator groups including: surface groups, Baumslag--Solitar groups, and in certain free-by-cyclic one-relator groups. We also prove results about the related positivity problem (also called the quasi-Magnus problem) which asks whether membership in the submonoid generate by $A$ is decidable. We resolve a problem posed by McCammond and Meakin in 2006 by showing that there is a hyperbolic group $G$ generated by $A$ such that it is undecidable whether an element can be represented by a positive word on the generators $A$. We show in addition this hyperbolic group can be chosen to be residually finite. We do this by giving a new general method for constructing finitely presented groups with undecidable positivity problem.