lemmydividebyzero@reddthat.com to Technology@lemmy.worldEnglish · 2 个月前Opensource AI Must Winopensourceaimustwin.comexternal-linkmessage-square111linkfedilinkarrow-up1286arrow-down120cross-posted to: [email protected]
arrow-up1266arrow-down1external-linkOpensource AI Must Winopensourceaimustwin.comlemmydividebyzero@reddthat.com to Technology@lemmy.worldEnglish · 2 个月前message-square111linkfedilinkcross-posted to: [email protected]
minus-squareshortwavesurfer@lemmy.ziplinkfedilinkEnglisharrow-up16arrow-down3·2 个月前Ollama.com Install it and then “ollama run olmo-3:7b” gets you a local AI. If you want to run a smarter AI, then you’re going to need a bigger parameter model, which is going to take more hardware to run.
minus-squaremabeledo@lemmy.worldlinkfedilinkEnglisharrow-up3·2 个月前Not many people can afford to run their own models.
minus-squareshortwavesurfer@lemmy.ziplinkfedilinkEnglisharrow-up5·2 个月前Not the big ones, no. But I think almost everybody can run one with seven billion parameters.
minus-squaremabeledo@lemmy.worldlinkfedilinkEnglisharrow-up1arrow-down1·2 个月前Considering that the vast majority of PCs in the world have 8GB of RAM at most, I still doubt it.
minus-squareshortwavesurfer@lemmy.ziplinkfedilinkEnglisharrow-up4·2 个月前I ran it on my 8GB RAM desktop.
minus-squaremabeledo@lemmy.worldlinkfedilinkEnglisharrow-up1·2 个月前I mean if that’s all that would be loaded in memory, sure.
minus-squareZephyrXero@lemmy.worldlinkfedilinkEnglisharrow-up4arrow-down1·2 个月前I got Qwen 3.5:9b running on my 8GB GPU the other day, and it still has some room left over
minus-squaremabeledo@lemmy.worldlinkfedilinkEnglisharrow-up2·2 个月前I was talking about combined system RAM. People often overestimate what the average system specs are.
minus-squareZephyrXero@lemmy.worldlinkfedilinkEnglisharrow-up4arrow-down1·2 个月前If the model dumps over to system ram it gets super slow, you ideally want it to fit completely in your VRAM
Ollama.com
Install it and then “ollama run olmo-3:7b” gets you a local AI. If you want to run a smarter AI, then you’re going to need a bigger parameter model, which is going to take more hardware to run.
Not many people can afford to run their own models.
Not the big ones, no. But I think almost everybody can run one with seven billion parameters.
Considering that the vast majority of PCs in the world have 8GB of RAM at most, I still doubt it.
I ran it on my 8GB RAM desktop.
I mean if that’s all that would be loaded in memory, sure.
I got Qwen 3.5:9b running on my 8GB GPU the other day, and it still has some room left over
I was talking about combined system RAM. People often overestimate what the average system specs are.
If the model dumps over to system ram it gets super slow, you ideally want it to fit completely in your VRAM