I did add the two that work to the distribution of possible youtube providers. I've realized a lot of normal youtube embeds are broken for people. Apparently scored has the same problem. Something to do with adblock? Invidious instances seem to be broken more often than not.
I'm currently A/B testing which hosts are the most reliable. If an youtube exapando doesn't open right (seems to be more the case than not), close and re-open it. It just might work. But more importantly that sends me a signal that the previous host sucked.
What sucks is I've been tracking these stats for a while and different invidious hosts wax and wane in their reliability. Right now invidious.nerdvpn.de is the most reliable one producing 10 re-try load events in 103. Normally inv.perditum.com is the most reliable but it produced 61 re-try load events in 350.
The selection of host it picks is randomized by a weighted distribution that makes selection of the best host likely but provides enough traffic to the others to help discover which is actually best (since it isn't a clear provider). Here is a our current list of providers in order of selection probability.
The formula used is invhosts[Math.round(-0.3*Math.log(Math.random()))%invhosts.length];
This is an exponential distribution using a hack I saw by Knuth in Art of Programming II.
As you can see plain Jane youtube is one of the providers so we can measure invidious being generally buggy vs youtube being consistently buggy with embeds for people with ad block while on a site that doesn't have an exception carved out for them (what might be happening to make it so Reddit doesn't have a problem). The youtube-nocookie.com is actually what reddit uses. But it is able to create the same error regular youtube does when embedding on a smaller site with ad block on.
So the reliability has something to do with server/client communication and not the server reliability itself? Or is this whole post saying you do not know?
It depends on which one. With the invidious I've notice multiple ways it can error out and it is 100% internal to invidious. I can't make these instances more reliable. But if one of them gets their act together I might be able to measure it.
For the youtube, I don't know. I know I'm not the only site with the issue. Youtube themselves recommends just doing /embed/{id} in an iframe. Seems simple. But then you get a "153 error: client misconfigured" at least while in Brave and I'm told even some people not in Brave have seen it. I and others have also seen it on other sites. Somehow reddit's embeds work. I've looked into what they are doing and it seems to be mostly the same and any difference I've tested and seen the same problems. So in that case it is a case of "I don't know." I've seen a hand wavy explanation online that it has to do with ad block.
These new piped providers I've added might just work. But I've played around with piped before and the experience was similar to invidious. You find an instance that just works, then it goes bad in two weeks. Maybe it was a mistake to add them to the bottom of the list because they aren't going to get as much data fed to them in the A/B test. But the A/B test is intended to be a meritocracy so if they work hopefully they rise to the top.
My hope is that by making this post people will intentionally close and re-open the expandos if they suck and give me more and better data instead of immediately giving up and going to youtube. Even if you do end up going to youtube when an expando doesn't work giving it a second try means I do get some data that the first host sucked.
Even if I did understand why invidious sometimes fails or why youtube always fails for some I'd still want a measure to compare them. So if I have to act on a metric anyway who cares to understand what drives it?
Even if you do end up going to youtube when an expando doesn't work giving it a second try means I do get some data that the first host sucked.
Might want to make that bold.
What if we started our own non-publicized instance for GM and friends? I have Chew.Tube and Volun.Tube to utilize. But first we'd have to know how much storage/bandwidth to find the co$t before we'd know if it was worth it or not. Also, it might be worth setting up more than one server to rotate through, and/or a VPN, so it's less traceable and/or avoiding centralized co$ts.
Also, it might be good to YT-DLP archive videos, then add an archive expando beside every YouTube expando. In addition to me eventually, I'd guess others would also be willing to co-host a distributed vault.
I was just wondering if me hosting some $10 hetzner server and putting a yt proxy on it would help. If it's the server's reliability and not the interconnect, I could partially alleviate the issue.
Well I know YouTube wants to make proxying their service difficult. You could do it with yt-dlp if you keep it very up to date.
In a way this is not just a goatmatrix thing but also a gvid thing.
Maybe the other answer is we all just need to ween ourselves off of YouTube because it doesn’t embed well in all browsers. It’s intentionally doing that to be an advertisement for Chrome. Then they carve out an exception for big sites so people are more likely to blame their browser or the forum instead of the origin site that is intentionally shitting the bed.
I've been looking at what it takes to host a piped+ip rotator, and it seems like an afternoon's worth of work. Sounds fun. Not exactly sure how you interface with them, but we can talk when the time comes.
Log in
I did add the two that work to the distribution of possible youtube providers. I've realized a lot of normal youtube embeds are broken for people. Apparently scored has the same problem. Something to do with adblock? Invidious instances seem to be broken more often than not.
I'm currently A/B testing which hosts are the most reliable. If an youtube exapando doesn't open right (seems to be more the case than not), close and re-open it. It just might work. But more importantly that sends me a signal that the previous host sucked.
What sucks is I've been tracking these stats for a while and different invidious hosts wax and wane in their reliability. Right now invidious.nerdvpn.de is the most reliable one producing 10 re-try load events in 103. Normally inv.perditum.com is the most reliable but it produced 61 re-try load events in 350.
The selection of host it picks is randomized by a weighted distribution that makes selection of the best host likely but provides enough traffic to the others to help discover which is actually best (since it isn't a clear provider). Here is a our current list of providers in order of selection probability.
The formula used is
invhosts[Math.round(-0.3*Math.log(Math.random()))%invhosts.length];This is an exponential distribution using a hack I saw by Knuth in Art of Programming II.
As you can see plain Jane youtube is one of the providers so we can measure invidious being generally buggy vs youtube being consistently buggy with embeds for people with ad block while on a site that doesn't have an exception carved out for them (what might be happening to make it so Reddit doesn't have a problem). The youtube-nocookie.com is actually what reddit uses. But it is able to create the same error regular youtube does when embedding on a smaller site with ad block on.
So the reliability has something to do with server/client communication and not the server reliability itself? Or is this whole post saying you do not know?
It depends on which one. With the invidious I've notice multiple ways it can error out and it is 100% internal to invidious. I can't make these instances more reliable. But if one of them gets their act together I might be able to measure it.
For the youtube, I don't know. I know I'm not the only site with the issue. Youtube themselves recommends just doing /embed/{id} in an iframe. Seems simple. But then you get a "153 error: client misconfigured" at least while in Brave and I'm told even some people not in Brave have seen it. I and others have also seen it on other sites. Somehow reddit's embeds work. I've looked into what they are doing and it seems to be mostly the same and any difference I've tested and seen the same problems. So in that case it is a case of "I don't know." I've seen a hand wavy explanation online that it has to do with ad block.
These new piped providers I've added might just work. But I've played around with piped before and the experience was similar to invidious. You find an instance that just works, then it goes bad in two weeks. Maybe it was a mistake to add them to the bottom of the list because they aren't going to get as much data fed to them in the A/B test. But the A/B test is intended to be a meritocracy so if they work hopefully they rise to the top.
My hope is that by making this post people will intentionally close and re-open the expandos if they suck and give me more and better data instead of immediately giving up and going to youtube. Even if you do end up going to youtube when an expando doesn't work giving it a second try means I do get some data that the first host sucked.
Even if I did understand why invidious sometimes fails or why youtube always fails for some I'd still want a measure to compare them. So if I have to act on a metric anyway who cares to understand what drives it?
Might want to make that bold.
What if we started our own non-publicized instance for GM and friends? I have Chew.Tube and Volun.Tube to utilize. But first we'd have to know how much storage/bandwidth to find the co$t before we'd know if it was worth it or not. Also, it might be worth setting up more than one server to rotate through, and/or a VPN, so it's less traceable and/or avoiding centralized co$ts.
Also, it might be good to YT-DLP archive videos, then add an archive expando beside every YouTube expando. In addition to me eventually, I'd guess others would also be willing to co-host a distributed vault.
I was just wondering if me hosting some $10 hetzner server and putting a yt proxy on it would help. If it's the server's reliability and not the interconnect, I could partially alleviate the issue.
Well I know YouTube wants to make proxying their service difficult. You could do it with yt-dlp if you keep it very up to date.
In a way this is not just a goatmatrix thing but also a gvid thing.
Maybe the other answer is we all just need to ween ourselves off of YouTube because it doesn’t embed well in all browsers. It’s intentionally doing that to be an advertisement for Chrome. Then they carve out an exception for big sites so people are more likely to blame their browser or the forum instead of the origin site that is intentionally shitting the bed.
That is, unless you think invidious is a better option.
I've been looking at what it takes to host a piped+ip rotator, and it seems like an afternoon's worth of work. Sounds fun. Not exactly sure how you interface with them, but we can talk when the time comes.
Yeah. I just need something that will let me do /embed/{id} and give back something reasonable. I don't need any api support or anything fancy.