[Looking for Testers] Got an old NVIDIA GPU with at least 2GB VRAM? Help us build a decentralized AI network!

Hi everyone!

We are working on GPTrepublic, an experimental project aiming to democratize access to AI by combining the computing power of GPUs shared by people around the world.

Instead of running an entire AI model on one powerful GPU, we distribute its layers across multiple computers, allowing even older GPUs to contribute.

We’ve already had encouraging results testing the system across several VMs, and now we’re looking for volunteers for our first public test on real hardware!

We want to test network latency, stability, automatic layer distribution, and how the network handles nodes joining and leaving.

Our long-term dream is to make powerful AI models freely accessible to everyone through shared computing resources.

Have an old NVIDIA GPU with at least 2GB VRAM? We’d love to have you as a tester!

Learn more and sign up: https://gptrepublic.org

Thanks! :slight_smile:

I think your idea lacks proper safety considerations and as such is unsafe by design.

Who is going to stop the bad actor from using it to cause harm? Is your model proven safe?

How many research scenarios did you do to verify that it won’t make the terrible mistake?

Where is that evidence?

You have an interesting last name.

(“Cherry City” ) is an Eastern European name of Slavic origin.

The idea of community will win so we can begin to consider the World Citizen approach with AI.

It is an interesting topic.

-Ernst03

Yes, on both accounts, name and community. Thank you.

What types of optimization have you done to get larger models to run on smaller systems? Or is this just assuming they are already optimized?

Alongside tokens/s, it would be useful to track completed requests and retries, so work lost when a node leaves isn’t counted as a successful contribution. How are you thinking about balancing contributed compute with access to the network? I’m exploring that exchange in Community Compute, still just a concept without a prototype, and would love to compare notes on what a small, fair pilot should measure.

Thanks for raising these concerns! I think they’re valid questions and definitely worth discussing.

First, I want to emphasize that GPTrepublic is still at a very early experimental stage. So far, we’ve only conducted small-scale tests using virtual machines, and that’s exactly why I’m currently looking for testers. We’re not launching a finished public AI service. We’re trying to understand whether this kind of distributed infrastructure can actually work on a real network of everyday GPUs.

I think it’s also important to distinguish between two different aspects of safety:

Model safety: We’re not developing or training a new LLM from scratch. The idea is to distribute existing AI models across multiple machines. Of course, using existing models doesn’t automatically make them safe, and their limitations and potential misuse still need to be considered.

Infrastructure security: This is something we’re directly responsible for. Protecting participants’ machines, isolating workloads, managing access, preventing abuse, and ensuring that sharing GPU resources doesn’t compromise users’ privacy or security are all important challenges.

I’m certainly not claiming that we’ve already solved these problems or that we have extensive safety research to back up such a claim. We’re at the beginning of the experiment, and there’s still a lot of work to do.

The first testing phase is intended to help us identify technical limitations, vulnerabilities, and unexpected behaviors before even considering a larger public network.

I’m curious, though: when you mention safety, are you mainly concerned about harmful outputs from the AI models, malicious use of the shared computing network, or the security of the computers contributing their resources?

These are different challenges, and I’d genuinely appreciate hearing more about your concerns.

Thanks again for bringing this up! Constructive criticism is especially valuable at this early stage. :slight_smile:

Good question! :slight_smile:

So far, our tests have been done with relatively small models. The main goal wasn’t to run huge models right away, but to see whether we could actually distribute a model’s layers across multiple machines and get the whole system working together.

And it worked! Of course, testing on a few VMs is one thing, while running the same system across real users, different GPUs, and home internet connections is a completely different challenge.

That’s exactly the purpose of our first public test. We want to understand how the network behaves in real-world conditions, identify the bottlenecks, and figure out what needs to be improved.

Our main approach is distributing model layers based on each node’s available VRAM, rather than trying to fit an entire large model onto a single GPU. Quantization and other memory optimizations are also important, but we’re not claiming to have invented a new compression technique.

The long-term goal is to gradually increase the size of the models we can run, step by step. Of course, how far we can go will depend heavily on how many participants and GPUs we manage to bring into the network.

So right now, the biggest challenge isn’t just technical. It’s also building a community large enough to make this experiment meaningful!

We’re still at the beginning, but hopefully we’ll find plenty of people with old GPUs willing to help us push the limits!

Thanks for bringing this up! It’s actually one of the things we’re still trying to figure out! :slight_smile:

Right now, we’re considering two approaches.

The first is keeping basic AI access free for everyone, while giving higher priority to users who contribute computing resources.

The second is a more direct token-based exchange: if you help the network process 1,000 tokens, you earn the right to generate 1,000 tokens yourself. Basically, contribute tokens, earn tokens!

Of course, since our system distributes individual model layers across multiple GPUs, we need to figure out how to measure contributions fairly. Processing tokens on one layer isn’t necessarily equivalent to processing them on another, especially with different models and hardware.

We’re still at a very early experimental stage, so we haven’t decided which approach is best. We want to use data from our first real-world tests to develop a system that’s as fair as possible.

I also completely agree about tracking successful requests, retries, and lost work. Those metrics will be essential.

I’d love to compare notes as both projects develop! We’re approaching distributed computing differently, but I think we have a lot of common challenges to solve. :slight_smile: