Some questions and because you don't actually understand, also, the answers.

• what does the LLM understand the context of, (other user's data owned by Twitch)
• How is the LLM fed that data? (You store it and feed it to the LLM)
• Do you use Twitch's data and its users data through an AI without their consent? (Most likely, yes)
• Do you have consent from the users to store 'facts' about them (You're pissy, so obviously not)
• Are you then storing that processed data? (Yes, you are, written to a file)
• Is the purpose this data processing commercial (Yes, it is, designed to increase viewer count for the user of this system - and before you retort "OMG it helps twitch too"... Uhm no, Twitch has the viewers if not watching him, watching someone else)

I mean yeah, it's a use case, but own up to the fact that you're wrong. Or be pissy. I don't care.

True.

Although in Germany for example it can also be an issue when recording. If you have a security camera pointed at a public space (that can include the sidewalk infront of your house), passersby can sue you to take it down and potentially get you fined. Even pretending to constantly record such an area can yield that result.

You build a system to identify everyone in the park and collect recordings of their conversations? Absolutely a problem, depending on the jurisdiction.

Literally not. The police use this right now to record your location and time seen using license plates all over the nation - with private corporations providing the service.

and being in public doesn't automatically entail consent to being recorded.

And yes, it's called 'expectation to the right of privacy'. Public venues are not 'private' locations, and thus do not need consent. You can, quite literally, record anyone in public.

Even the link you provided agrees.

There's not actually that much code. It's like 8 lines for an AI 'agent', and maybe another 16 lines for 'tools', and I'm using Streamlink for grabbing the audio stream, and pulseaudio has a 'monitor' device you can use to listen to what's playing on the speakers. Throw it on a very minimal linux distro on a VM, and that's it.

I don't do 'vibe coding', but that IS where I got the idea from. People who are doing 'vibe coding' nowadays aren't just plugging things into a generic AI, they're spinning up 'agents' and making tools via MCP and then those agents are tasked with specific things, and use the tools to directly write to files, search the internet, read documents, etc

I've tried coding and every one I've tried fails unless really, really basic small functions like what you learn as a newbie compared to say 4o mini that can spit out more sensible stuff that works.

I've tried explanations and they just regurgitate sentences that can be irrelevant, wrong, or get stuck in a loop.

So. what can I actually use a small LLM for? Which ones? I ask because I have an old laptop and the GPU can't really handle anything above 4B in a timely manner. 8B is about 1 t/s!

Please be very careful. The python code it'll spit out will most likely be outdated, not work as well as it should (the code isn't "thought out" as if a human did it.

If you want to learn, dive it, set yourself tasks, get stuck, and f around.

So this wasn't a post actually asking what a small LLM was good for, it was just an opportunity you could use to dump on LLM usage I take it. So this whole thing was made in bad faith?

With the comments about "vibe coding" and such, all it looks like you're doing here is arguing the "merits" of how it's being used, and you're not interested in its actual usage at all.

Nobody is being pissy here except you. Small LLMs can be used for tasks such as this, and it doesn't have to be twitch - It could be an assistant that you build for reminders in your personal life - using it on twitch is a minor detail that you seem to have latched onto because you just want to dump on LLM usage.

Go to /c/fuck_ai for that.

I gave you an example that it's good for, and all you want to do is argue the merits of how I'm using it (even though it falls perfectly within Twitches TOS and use cases)

I've tried coding and every one I've tried fails unless really, really basic small functions like what you learn as a newbie compared to say 4o mini that can spit out more sensible stuff that works.

I've tried explanations and they just regurgitate sentences that can be irrelevant, wrong, or get stuck in a loop.

So. what can I actually use a small LLM for? Which ones? I ask because I have an old laptop and the GPU can't really handle anything above 4B in a timely manner. 8B is about 1 t/s!

I've tried coding and every one I've tried fails unless really, really basic small functions like what you learn as a newbie compared to say 4o mini that can spit out more sensible stuff that works.

I've tried explanations and they just regurgitate sentences that can be irrelevant, wrong, or get stuck in a loop.

So. what can I actually use a small LLM for? Which ones? I ask because I have an old laptop and the GPU can't really handle anything above 4B in a timely manner. 8B is about 1 t/s!

for coding tasks you need web search and RAG. It's not the size of the model that matters, since even the largest models find solutions online.

If I say my name is Doo doo head, in a public park, and someone happens to overhear it - they can do with that information whatever they want. Same thing. If you wanna spew your personal life on Twitch, there are bots that listen to all of the channels everywhere on twitch. They aren't violating any laws, or Twitch TOS. So, *buzzer* WRONG.

Right now, the same thing is being done to you on Lemmy. And Reddit. And Facebook. And everywhere else.

Look at a bot called "FrostyTools" for Twitch. Reads Twitch chat, Uses an AI to provide summaries of chat every 30 minutes or so. If that's not violating TOS, then neither am I. And thousands upon thousands of people use FrostyTools.

I have the consent of the streamer, I have the consent of Twitch (through their developer API), and upon using Twitch, you give the right to them to collect, distribute, and use that data at their whim.

You're both getting side-tracked by this discussion of recording. The recording is likely legal in most places.

It's the processing of that unstructured data to extract and store personal information that is problematic. At that point you go from simply recording a conversation of which you are a part, to processing and storing people's personal data without their knowledge, consent, or expectation.

@kattfisk That seems to imply that you cannot personally listen to or watch recordings that you have made in public. In doing so, you are abstracting personal details that you might have missed before, refreshing your memory, and so on. What is the material difference between you doing this without machine help versus with automation that makes it ethically problematic? What if a friend helped you, not a machine?

I just use the companion app for now. But I am designing a HAL9000 system for my home.

[ A DIM SCREEN WITH ORANGE TEXT ]

Objective: optimize electrical bill during off hours.

... USER STATUS: UNCONSCIOUS 
... LIGHTING SYSTEM: DISABLED
... AUDIO/VISUAL SYSTEM: DISABLED 
... CLIMATE SYSTEM: ECO MODE ENABLED
... SURVEILLANCE SYSTEM: ENABLED 
... DOOR LOCKS: ENGAGED
... CELLULAR DATA: DISABLED
... WIRELESS ACCESS POINTS: DISABLED
... SMOKE ALARMS: DISABLED
... CO2 ALARMS: DISABLED
... FURNACE: SET TO DIAGNOSTIC MODE
... FURNACE_PILOT: DISABLED
... FURNACE_GAS: ENABLED

WARN: Furnace gas has been enabled without a Furnace pilot. Please consult the user manual to ensure proper installation procedure.

... FURNACE: POWERED OFF

Objective realized. Entering low power mode.

[ Cut to OP, motionless in bed ]

Luckily my entire neighborhood doesn't have gas and I have a heat pump.

But rest assured, I'm designing the system with 20% less mental illness

Let's take a look at the Developer Agreement that you cited:

You must only retain chat logs as long as necessary for the operation of Your Services or to improve Your Services; do not do so for the purpose of creating public databases or websites, or, in general, to collect information about Twitch’s end users. You must enable, and process, all requests by end users to block, discontinue, delete, or otherwise opt-out of any retention of chat logs for Your Services.

This very clearly states that you are disallowed from retaining chat logs for the general purpose of collecting information about Twitch's end users.

You said that you, "store ‘facts’ about specific users so that they can be referenced quickly," but then later in a different thread state, "I’m not storing their data. I’m feeding it to an LLM which infers things and storing that data." You're retrieving information about specific users at a later time. You've built a database of structureless PII from chat logs. You've chosen to store the data as inferences, which makes it a bad database, but still a database.

I have questions:

When your streamer mentions something deeply personal, like, "how their mothers surgery went," that your tool helped them remember, do they disclose that your tool was involved in that transaction?
When the viewer gets weirded out and asks your streamer to not mention that again, or forget it entirely, do you have a way to remove that information from your database and a way to prove it's been deleted?
When other people in chat think it's gross, and ask to opt-out, can you even do it?

Regarding FrostyTools: I don't think it's storing the chat logs for a later time. They don't have a data retention section in their TOS or Privacy Policy that isn't related to the streamer. (As in, they hold on to the streamer's Twitch account and some other information for billing, authentication, etc.) I think it's taking the chat logs only for as long as it needs to output a response and then deleting it. Also, this excerpt from the FrostyTools TOS made me chuckle:

This means that you, and not FrostyTools, are entirely responsible for all Content that you upload, post, email, transmit, stream, or otherwise make available via the Service. FrostyTools does not control the Content posted via the Service and, as such, does not guarantee the accuracy, integrity or quality of such Content. You understand that by using the Service, you may be exposed to Content that is offensive, indecent or objectionable. Under no circumstances will FrostyTools be liable in any way for any Content, including, but not limited to, any errors or omissions in any Content, or any loss or damage of any kind incurred as a result of the use of any Content posted, emailed, transmitted, streamed, or otherwise made available via the Service.

You agree that you must evaluate, and bear all risks associated with, the use of any Content, including any reliance on the accuracy, completeness, or usefulness of such Content. In this regard, you acknowledge that you may not rely on any Content created by the Service or submitted to the Service.

This leads me to believe that you can violate the Twitch TOS quoted above using FrostyTools. It is apparent that FrostyTools has positioned itself as an application that creates User Generated Content (like Photoshop or Word).

All systems need a little mental illness.

Any suggestions for solutions?

Any suggestions for solutions?