Lightnews — Scholar-powered news

boo-liet shen 👻 @julietshen.bsky.social · 22m

don't get me started on dating apps + mod tool access, I always feel like they're not held to the same standards and discourse as social media

There are a few public examples of this too I think, might be part of the tools paper we wrote

3

boo-liet shen 👻 @julietshen.bsky.social · 39m

also they (like pretty much everyone) use a 3rd party age verification software vendor! one can only be as secure as one's vendors are~

2

boo-liet shen 👻 @julietshen.bsky.social · 2h

Trust & safety and content moderation are at their purest forms challenges in scaling.

Both the computational cost of scaling in terms of raw volume AND scaling accurate and consistent decisions. The latter is extra hard cause people are unpredictable and human behavior is weird and ever changing

hailey @hailey.at · 12h

an interesting note re: scaling image moderation. in the past 24 hours:

at least 500 thousand images in posts (not considering # of images, just if an image was in the post)
86 thousand avatars
500 thousand url thumbnails
58 thousand videos.

that's a lot to moderate!

10

Reposted by boo-liet shen 👻

Aaron Rodericks @aaron.bsky.team · 6h

I'll be trying out leaflet.pub moving forward to try to write longer form blogs on Trust and Safety so that I can explain things in a bit more depth.
leaflet.pub/25030fdf-2a8...

Context Collapse & T&S

One of the hardest things about writing on a microblogging service is context collapse, especially when it comes to complex issues — you reply to one person, others take it out of context, and suddenl...

leaflet.pub

16 14 81

boo-liet shen 👻 @julietshen.bsky.social · 2h

This is cool

techcrunch.com/2025/09/30/h...

Hinge is taking a fairer approach to account banning | TechCrunch

Hinge won’t kick you off the app if just one thing on your profile breaks the rules but it does remove the content and prompts you to change it.

techcrunch.com

3

Reposted by boo-liet shen 👻

rahaeli @rahaeli.bsky.social · 18h

For as much as Hive has built-in biases (and every model has built-in biases, because they're trained on human data and humans have built-in biases) it is, hands down, one of the more accurate commercially available services out there. This doesn't mean it doesn't suck on a few things!

1 3 43

Reposted by boo-liet shen 👻

rahaeli @rahaeli.bsky.social · 18h

Absolutely everyone knows that these available services suck at a few things! But they do refine over time (my favorite example of an image Hive in particular used to detect: bsky.app/profile/raha... )

rahaeli @rahaeli.bsky.social · 19h

The example I always use: earlier generations of Hive's model used to detect this as "adult content"! (My best guess is the "kinda human skin colors" + "rounded shadows like underboob shadows" of rows 1+2 are right where boobs would be in a closeup nude.)

An image of 8 rounded swatches, shaded like they're raised, on a color card. The first and second row are colors that could possibly be "human skin tones" and the shadows under each swatch are darker, curved, and rounded.

1 3 42

Reposted by boo-liet shen 👻

rahaeli @rahaeli.bsky.social · 18h

github.com/bluesky-soci... -- even if you can't really fluently read code, you can still see that there's a bunch of specialcasing where they're combining two scores to be like "hey, Hive kinda sucks at this stuff so let's increase the confidence threshold"

indigo/automod/visual/hiveai_client.go at 3259b215110eb8bffcd26fc340203f828e6dadad · bluesky-social/indigo

Go source code for Bluesky's atproto services. Contribute to bluesky-social/indigo development by creating an account on GitHub.

github.com

1 2 40

Reposted by boo-liet shen 👻

rahaeli @rahaeli.bsky.social · 18h

You can see the full list of what the API returns here: docs.thehive.ai/docs/visual-... Sites then take those scores and decide at what threshold to take action. Like, a site that doesn't care about depictions of drug use might completely ignore that category of labels, etc.

Visual Moderation - Overview

Introduction V3 (Demo) vs V2 (Enterprise) at a glance V3 API — easiest to try today in the Playground . For developer testing ONLY : 100 requests/day limit. Great UI for testing and proofs-of-concept....

docs.thehive.ai

1 3 33

Reposted by boo-liet shen 👻

rahaeli @rahaeli.bsky.social · 18h

The way Bluesky's implementation of Hive's service works is they query Hive's backend automated interface (API) for every uploaded image, and the API returns confidence scores from 0-1 based on "how much the model thinks this thing matches the criteria for this element".

1 1 37

boo-liet shen 👻 @julietshen.bsky.social · 17h

Rudy is the GOAT

and this kind of collective pluralism is what makes his work a real revolution

Rudy wants revolution. @rude1.blacksky.team · 17h

If this becomes a big enough problem, we can switch gears over time! When Blacksky Private Posts ships (eventually) that won't touch Bluesky services at all and we (the Blacksky community) can decide collectively how we want that to work (maybe we don't allow images/videos at all!) 10/11

1 9

Reposted by boo-liet shen 👻

Rudy wants revolution. @rude1.blacksky.team · 17h

P.S. I've been updating docs.blacksky.community based on questions and feedback so check that out.

Thanks again to the folks who trust us and contribute the funds that allow to continue pursuing this mission of providing communities with autonomous, safe spaces.

Gonna get back to work now 👨🏾‍💻

Introduction | Blacksky Documentation

docs.blacksky.community

35 130 840

boo-liet shen 👻 @julietshen.bsky.social · 18h

going to bed before I post something weird from the wrong account

5

boo-liet shen 👻 @julietshen.bsky.social · 18h

if you know solid software engineers who want to be part of our open source software adventure, email [email protected]

+10 if they've worked on trust & safety
+10 if they have worked with graphql and/or typescript
+100 if they like chicken puns
+1000 if they want to change the world

4 8 43

boo-liet shen 👻 @julietshen.bsky.social · 18h

we hirin

Anne Bertucio @annebdh.bsky.social · 18h

Want to work on open source full time? The @roost.tools engineering team is starting to hatch! Come build OSS tools making a difference in Trust & Safety.

This is a fully remote role, though some schedule overlap with North American time zones is expected.

www.linkedin.com/jobs/view/43...

ROOST.tools hiring Staff Software Engineer in United States | LinkedIn

Posted 8:52:18 PM. About ROOSTROOST is a community effort to build scalable and resilient safety infrastructure for…See this and similar jobs on LinkedIn.

www.linkedin.com

1 2 14

Reposted by boo-liet shen 👻

Anne Bertucio @annebdh.bsky.social · 18h

Want to work on open source full time? The @roost.tools engineering team is starting to hatch! Come build OSS tools making a difference in Trust & Safety.

This is a fully remote role, though some schedule overlap with North American time zones is expected.

www.linkedin.com/jobs/view/43...

ROOST.tools hiring Staff Software Engineer in United States | LinkedIn

Posted 8:52:18 PM. About ROOSTROOST is a community effort to build scalable and resilient safety infrastructure for…See this and similar jobs on LinkedIn.

www.linkedin.com

3 25 54

boo-liet shen 👻 @julietshen.bsky.social · 18h

1000000%

boo-liet shen 👻 @julietshen.bsky.social · 18h

Thank you! I heard they had to stop the labeler, so even though I had liked it that's why I never got sorted 🥺

1 1

boo-liet shen 👻 @julietshen.bsky.social · 19h

this part of France is soooooooooo pretty

I love remote work

A view on the walk up to éze in southern France with a temperature sticker showing 21°C

A medieval stone wall covered in ivy in shades of green, orange, and deep red with a french flag blowing in the wind. The sky is blue with some wispy white clouds

A view of the French medieval village with a soft golden glow from the sun. The walls are old and a mix of large stones and smoother surfaces. The doors on the buildings are a dark brown wood, as are the old window shutters. There is a black metal gate in front of one open doorway. The stairs have a red paved path in the middle.

A hand stretches out and cradles a pomegranate on a tree. It is large but not ripe, with a gradient of yellow-green and red. Another smaller pomegranate hangs nearby on the same tree.

1 18

boo-liet shen 👻 @julietshen.bsky.social · 19h

i was using my own knowledge and experience with Hive (which apparently is outdated, probably also because I'm deep in T&S but not being in the loop on their weird flashy genai features), but recognize how that came across condescending - sorry about that!

hope you have a good rest of your day~

1

Reposted by boo-liet shen 👻

rahaeli @rahaeli.bsky.social · 21h

This is known as "reinforcement learning from human feedback" (en.m.wikipedia.org/wiki/Reinfor...). It is very complicated and there are upsides and downsides that are beyond the scope of what I have the capacity to explain right now, but it's how you train unwanted outcomes out of a model.

Reinforcement learning from human feedback - Wikipedia

en.m.wikipedia.org

1 9 200

Reposted by boo-liet shen 👻

rahaeli @rahaeli.bsky.social · 21h

Also, you can't be simultaneously upset that the ML classifier gets things wrong and be upset at the thought of training on user data, because the two are mutually exclusive. The only way to get more accurate ML models is to retrain with real-world examples.

3 19 300

boo-liet shen 👻 @julietshen.bsky.social · 19h

hi hi, i'm really trying to engage in good faith but i'm getting a weird vibe from this reply :( can we try to talk to each other for real?

i appreciate the correction though!

1

boo-liet shen 👻 @julietshen.bsky.social · 19h

i cant speak for bsky or hive but i've been on the customer side of similar companies before, and this is usually covered in the contract. Companies can dictate whether their data is used to train genAI, and i THINK bluesky has said a few times that they absolutely do not do this with any vendors

1

boo-liet shen 👻 @julietshen.bsky.social · 19h

PS i have been corrected by @verdverm.com !

bsky.app/profile/verd...

verdverm @verdverm.com · 19h

Hive does have Gen AI products now

thehive.ai/apis/image-g...

Image Generation APIs to generate unique digital images with text prompts | Hive

Generate images using text prompts

thehive.ai

1