The fixed supply of land
It's not 100% accurate to say that no land is withdrawn from supply with high land taxes - it would certainly disincentivise land-owners from funding resource and mineral prospecting on their land which could be considered a form of "land creation". It also disincentivises land-owners from seeking land use re-designations (from agricultural use to residential use say). And I suppose that if someone owned a shallow seabed at the coast that could be reclaimed from the sea given investment, then a land tax would make this investment less profitable. Bryan Caplan raises this issue in A Search-Theoretic Critique of Georgism.
It can be claimed that taxing land taxes positive spillovers created by other private investment (your land value depends positively upon how nice your neighbours land is and upon private amenities close by) and that this may reduce the supply of such investment: The Problem With 100% Land Value Taxes. The anecdotal example given to describe the issue is "Real estate developers who move into neighborhoods with high vacancies, low demand, and high crime are often hoping that positive spillovers from their investment will spur additional investments from others, which will in turn make their investment more valuable." Noah Smith disagrees that land taxes are an issue here: "the problem of neighborhood externalities is a thorny one, but the LVT does not make it worse (or better)".
Land as an asset class
In an OLG model, the introduction of land crowds out productive capital formation - see Deaton & Laroque (2001)
Sounds good – let’s go
Chris Dillow posts a depressing Venn Diagram in The Economic Policy Dilemma:
Finally, worth posting another link to Noah Smith's fantastic summary: This 100-year-old idea could end San Francisco’s class war