(How to) transform a playing field into a great map?

[Schlechtwettermonste]

Hi. I have developed a generator that creates a map that defines the terrain of a playing field in 30x20 squares. I think you can guess what the symbols and colors mean. Example:

The map is generated anew each time, so it shouldn’t repeat itself.
I’ll need a minimap for the briefing later on, though the map doesn’t look all that great—it’s rather informal.

I also have a style sheet that is to be used:

The map should look like this, but with the terrain information passed from the generated map.

The AI ​​needs to run locally on a computer and be able to complete the task within 1–2 minutes.
I currently use ComfyDesktop with SDXL and IPAdapter for this:

As you can see, despite the precise task description in the positive prompt, the result is still far from matching the map in terms of both quality and accuracy:

The map needs to be quite precise. Currently, it consists of a 30x20 grid
(though there will likely be more squares later); the properties of these squares, as well as the course of the river, etc., must be clearly and accurately represented.

I still have the map in a sort of text format—basically just as the computer generated it—though I’m not sure if that helps?:

Could someone help me figure out how to implement this?
That would be absolutely fantastic!

Best regards,
Toby

Edit: By the way, this is what ChatGPT conjured up from my map template:

I was busy and didn’t notice it, though I assume it took longer than two minutes—albeit not much longer. In principle, however, it clearly demonstrates that it is possible.

[Lance-Smith]

reason one of the map images seems mushy is your running out of memory in late stage detailing. atleast thats how i describe it.
my recomendation - break up map creation in to 2 componets

1. map architecture
2. Map Art.

lets those be 2 different cycles. that way you get full generative capacity on the detail stage of creation. see if that helps your workflow.
i love the project though. i have a project for a TTRPG engine, and something like this is a great related concept.

[Schlechtwettermonste]

I’ve ultimately decided to generate the map myself using a script. I tried out several AIs, but they either failed to deliver (many struggled with the perspective alone) or produced results of insufficient quality.

I do have another AI up my sleeve for the game, though it serves an audio-related function (no, not music).

As for the solution regarding this thread:
I experimented with masks, height maps, color maps, style templates, and so on. However, the various workflows never yielded the results I had hoped for.
Ultimately, the required output was too specific and rigid for an AI to handle with the necessary quality and high level of accuracy (>99%).

Training a custom, lightweight AI would have been an alternative, but for a map that only exists in the game for about 30 seconds, that level of effort simply isn’t worth it.