I’ve tried to add a persistance storage to my space.
I’m running Flowise using a dockerfile, but i cannot get the storage to persist my sqlite setup.
hi @Academondo ,
Do you know to customize the working dir with Flowise? on HF the persistent data is always mounted as volume /data If you can point Flowise to write and read the volume that would probably make it work.
Thank you for your reply.
i use this dockerfile. but nothing is persisted it seems.
FROM node:18-alpine
USER root
Arguments that can be passed at build time
ARG FLOWISE_PATH=/usr/local/lib/node_modules/flowise
ARG BASE_PATH=/data/.flowise
ARG DATABASE_PATH=$BASE_PATH
ARG APIKEY_PATH=$BASE_PATH
ARG SECRETKEY_PATH=$BASE_PATH
ARG LOG_PATH=$BASE_PATH/logs
ARG OVERRIDE_DATABASE=false
Install dependencies
RUN apk add --no-cache git python3 py3-pip make g++ build-base cairo-dev pango-dev chromium
ENV PUPPETEER_SKIP_DOWNLOAD=true
ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser
ENV HF_HOME=/data/.huggingface
Install Flowise globally
RUN npm install -g flowise
Configure Flowise directories using the ARG
RUN mkdir -p $LOG_PATH $FLOWISE_PATH/uploads && chmod -R 777 $LOG_PATH $FLOWISE_PATH
WORKDIR /data
CMD [“npx”, “flowise”, “start”]
hi @Academondo , I made a couple of changes here https://huggingface.co/spaces/radames/Flowise/blob/main/Dockerfile this seems to be working now
I tested building and rebooting, the Space, the sqlite file is now saved and reload correctly
FROM node:18-alpine
USER root
# Arguments that can be passed at build time
ARG FLOWISE_PATH=/usr/local/lib/node_modules/flowise
ARG BASE_PATH=/data/.flowise
ARG DATABASE_PATH=$BASE_PATH
ARG APIKEY_PATH=$BASE_PATH
ARG SECRETKEY_PATH=$BASE_PATH
ARG LOG_PATH=$BASE_PATH/logs
# Set the ARGs as ENV
ENV FLOWISE_PATH=$FLOWISE_PATH
ENV BASE_PATH=$BASE_PATH
ENV DATABASE_PATH=$DATABASE_PATH
ENV APIKEY_PATH=$APIKEY_PATH
ENV SECRETKEY_PATH=$SECRETKEY_PATH
ENV LOG_PATH=$LOG_PATH
# Install dependencies
RUN apk add --no-cache git python3 py3-pip make g++ build-base cairo-dev pango-dev chromium
ENV PUPPETEER_SKIP_DOWNLOAD=true
ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser
# Install Flowise globally
RUN npm install -g flowise
# # Configure Flowise directories using the ARG
RUN mkdir -p $FLOWISE_PATH/uploads && chmod -R 777 $FLOWISE_PATH
CMD ["sh", "-c", "mkdir -p $LOG_PATH && chmod -R 777 $LOG_PATH && npx flowise start"]
1 Like
Hi @radames, this seems to be working! Care to elaborate why the last part where you create the $LOG_PATH directory only works when using sh?
Hi @station-dev, of course, Flowise was looking for $LOG_PATH in order to initialize. This is where it stores the persistent data. As I mentioned before, the /data/ volume is only mounted at runtime. Therefore, before running Flowise, it creates the $LOG_PATH folder at runtime.
1 Like
hi, since huggingface space doesn’t have persistent storage in free tier, is it possible to add our own 3rd party database for flowise?
hi @ryansmith69 , if you can have flowise remotely connect to the external database, then it’s possible, We have many Spaces, talking to external SQL databases.
@radames Hi, I tried to do it like they have written in the Flowise Docs but, I can’t connect my database with Flowise. Every time I restart space the chatflows, credentials, tools get deleted!
@radames Can you please show me how to do that?
@ryansmith69 Have you done it successfully?
hi @sarahcarlson, I’m not familiar with Flowise, but reading their docs here it looks like this option is enabled, so on every launch or Space restart your DB is recreated?
Synchronize in Production
Flowise uses Typeorm to configure database connection. By default, synchronize is set to true. This indicates if database schema should be auto created on every application launch.
However, we have to be careful with this option and don’t use this in production - otherwise you can lose production data. This option is useful during debug and development.
To override the value, set the following env variable
@radames well, I set all database credentials but, it still doesn’t work. do I need to change database path, apikey path, secretkey path and all these path from /root/.flowise to something else? if so, what do I set them as?
@radames How bout you just show the steps like you showed earlier about the hugging face persistent disk. you’re an experienced man so you can help me. the flowise support team is crap so, there is nowhere to seek for support.
hi @sarahcarlson ,
I did a fresh new duplication and set up a PostgreSQL with https://supabase.com/. After reboot it successfully reload the last state from the external DB.
You can modify your Dockerfile to include
ENV DATABASE_TYPE=postgres
ENV DATABASE_PORT=5432
ENV OVERRIDE_DATABASE=false
And add these as Secrets
DATABASE_TYPE=postgres
DATABASE_PORT=5432
DATABASE_HOST=localhost
DATABASE_NAME=flowise
DATABASE_USER=user
DATABASE_PASSWORD=123
FLOWISE_SECRETKEY_OVERWRITE= #Random encryption key to recover tokens from the DB
Additionally, you can duplicate this new example Flowise DB Example - a Hugging Face Space by SpacesExamples. When you use this example, you will be prompted to enter the required variables and secrets.
By the way, do I need to change the database path and also I have to paste these in dockerfile. right?
ENV DATABASE_TYPE=postgres
ENV DATABASE_PORT=5432
ENV OVERRIDE_DATABASE=false
So sorry to bother you but, where do i get FLOWISE_SECRETKEY_OVERWRITE or can i do i create it in my own way like, whatever i like.
You can generate a random string. This is used to encrypt the secret tokens inside the database, and it’s required when you reboot your Space.
please change the type and port in case you’re using MySQL, or your DB port is not the standard ports
