The real jury–televote split, v2: A post-analysis

In my previous post, I wrote up a big long analysis covering why the televoters have lost most of their power in Melfest since the app votes come in. Firstly, I mentioned that the last couple of years had been at a power split of around 24-76 in favour of juries. Then, I noted that the changes to the jury votes were expected to help fix that a little, but wouldn’t be perfect, lifting the power split to around 30-70, still favouring juries. So how did that work out, in the end? Well, in a sense, I was wrong…

…because the changes were even more miniscule than I’d expected. After plugging this year’s televotes and jury votes back into my formula, the effective power split for this year was: 24.6-75.4. To put this in context with previous years:

Power Split (Actual 2018)

And so the split moved by just a single percentage point in the favour of televoters. Wow. As a reminder, I’d expected that the jury changes would bring televote power back to around 30% this year, and already said that that would be a crappy outcome in need of major fixes from SVT. With that in mind, seeing things essentially stay unchanged is pretty laughable, to be honest. As for why the changes were even less useful than expected, I’ve got a few theories, and I expect that the full answer is a bit of a combination of each:

1. The juries agreed on their lower points more than expected. In trying to predict how the juries allocated their lower scores (9th and 10th places, which they hadn’t previously had to vote for), I assumed there’d be at least a little disagreement, with some unpopular songs picking up points here and there that they otherwise wouldn’t have. Instead, a good chunk of these extra points went towards the songs that were doing well anyway, leaving the least favourite acts of the jury vote – Mendez in particular – even further behind than they would’ve been otherwise. In this sense, the change intended to restore some power to televoters may have in some cases given the juries even more ability to kill off songs they don’t like.

2. The app votes caused even more clumping this year. Despite the app votes already forming into a little clumped ball of uselessness in previous years, we could at least see that Frans was a clear favourite in 2016, and that Nano enjoyed a semi-decent lead in 2017. This year may have been our first real year with no clear televote favourite – and when the televote was already clumped in years with a favourite, it serves to reason that it’d turn out even worse when there’s little difference in opinion to begin with.

3. Televote lines stayed open while the jury votes were revealed. Based on how long the app heart beat at full speed for each act, Mendez won the televote as far as app votes were concerned (2m15s beating heart), followed very closely by John Lundvik (2m14s). There was then a bit of a gap, followed by a close grouping of Liamoo, Benjamin, Felix, Samir & Viktor and Mariette (all between 2m and 1m53s). But once the app votes closed and it came to phone-voting time, televoters could see who the juries were voting for, and as such knew which acts would be wasted votes.

I doubt it’s a coincidence that, following the reveal of jury results, televotes stopped coming in as quickly for previous televote favourites Mendez, John, Liamoo, Samir & Viktor and Mariette, who by that stage were all at least 48 points behind and thus seemed like a lost cause. In response, Benjamin and Felix rose up the televote ranks, as they were seen as the only possible winners by then. With the way the preferences were arranged this year, that meant the televoters stopped voting for what had been their favourites, and instead focused on those that had been 4th and 5th in the app vote, largely equalising the top set of scores even more closely. It also means that the televotes may have in effect had less influence than the 24.6% I’ve calculated, since on top of that figure there was a section of voting where all televoters were essentially encouraged to vote only for the two jury favourites.

Without the juries having their direct votes and this influence on the televotes, I think it’s entirely possible, even likely, that Mendez would’ve been Sweden’s representative for the year. Whatever your opinion might be on his song and its chances at Eurovision – and I personally think it would’ve struggled as a result of his vocals, with Benjamin being a safer pick – it seems rather silly, in the context of Melfest also being a show meant for entertainment in its own right, to have a song with that sort of public support ending dead last on the scoreboard.

Photo from Wiwibloggs

The combination of those factors seems to have counteracted the change that was supposed to help the televotes, and so in the end juries once again had essentially three times the influence of televoters. Finally, I thought it’d be interesting to end on a few facts about the split to provide some alternative ways of looking at things:

  • Last year, third place in the televote won Melodifestivalen overall. This year, third place in the televote came last overall. The only real difference here was that the jury liked one and not the other; essentially, they can have a song finish first or last, wherever they want, largely regardless of televoter preferences.
  • The gap between first and last in the televotes (30 points) was far smaller than the gap between first and third (48 points) in the jury votes. Meanwhile, the gap between first and third in the televotes was only 5 points – barely a tenth that of its jury equivalent.
  • Adding up the differences between each voter group’s result and the final result (e.g. juries put a song 5th and it instead came 7th = 2 ranks), the juries only missed out on their perfectly-ordered final result by 10 ranks. Televoters missed out by 24 ranks.

Here’s hoping SVT finally makes some changes for next year.

The real jury–televote split, v2: A post-analysis

Some Stats on the Melfest App Vote: The Real Jury–Televote Split

The voting system in Sweden’s Melodifestivalen national final is supposedly a 50/50 split, with jurors and televoters giving out an equal number of points. But when the two groups use entirely different systems to award their votes, do things really play out so evenly in the show itself?

In a post I wrote on a fan forum the other day, I tried to explain why and how the app vote is hurting televote power at Melodifestivalen, turning the finals into a jury-dominated affair. Now, however, I’ve spent some time going over the stats to try and prove things a little more conclusively, and to show what the app vote is doing to the vote split in Sweden.

To give a bit of a frame of reference, I’ve looked over the split jury/televote scores for 2011-2015 (following the start of the international juries, but before the introduction of the app vote in finals) and 2016-2017 (after the app vote’s introduction). I’ve then taken a bit of a look at the changes set to come in this year to see what effect, if any, they’ll have on changing the effective jury–televote split.


What’s the app vote done so far?

First, I gathered all of the televote and jury vote scores from the 2011-2017 period, then found the standard deviation data for each using Excel. I’m no expert statistician, but this seemed like a reasonable way to measure voting power: if a set of votes rates higher for standard deviations, it means there’s more distance between each song’s total score, and therefore that said set of votes has had a larger influence on the final result. As such, I considered this figure to represent voting power.

Next, I compared this data between the jury and the televote for each year. As an example, if the jury votes came in at 20 standard deviations for a year and the televotes were at 30, then I took that to mean that the televotes were spaced out by 50% more than the jury votes, and therefore held 50% more voting power, giving them a 60-40 split in their favour. Repeating this for each of the seven years in question gave the following results:

I think this is a fairly clear representation of how, for the last couple of years, the jury vote has massively outpowered the televote. Previously, the televote’s proportional system gave it a slight edge over the juries, averaging out to about a 55–45 effective split… but as soon as the app votes were introduced, the televotes clustered and lost their power, making the split roughly 24-76 in favour of juries. This is similar to the conclusion of a 22-78 split that I reached in my more simple earlier post.

Seeing the relative consistency in the split both before and after the app vote introduction shows that this is obviously no coincidence. For five years in a row, the televote’s power hovered at a steady fifty-something percent – and then, immediately following the change, plunged to sub-25% for two years running. But what about 2018? Well, that brings us to the next question…


Are the changes to jury votes going to help with this?

In an effort to return some power to the televoters, SVT has announced a slight change to this year’s voting system. While juries previously awarded points to the twelve songs in a format of 12-10-8-6-4-2-1, they must now spread their votes more evenly across the field by using the standard Eurovision system (the televotes are then scaled up linearly, by about 35%, to remain nominally even with the total jury vote). Here’s another way to look at it:

Since the jury has to award more points overall but can’t give any more than previously to their top favourites, this change forces them to score more evenly across the board, partially replicating the effect that the app vote has on the televotes. But is it enough?

To find out, I converted the 2016-2017 jury scores into the new 2018 system (2011-2015, which had no app-vote in the finals, was left as-is to allow for comparison). This involved simply converting up for the most part (e.g. 6s became 7s, 4s became 6s etc. as per above), but there were also three new entries that had to be awarded points per jury – the three that were just below the cut-off line for points, scoring 8th, 9th and 10th.

Obviously, we have no information on what these scores would have been, so I’ve filled them in semi-randomly while favouring the more successful entries. The goal here was to replicate as closely as possible what the official results would’ve been under the 2018 system. As such, my approach was to assume that e.g. an overall jury second-placer would be more likely to get points than an overall tenth-placer, but also that not every last-place entry would be in last with every juror.

Combining these new converted scores with our previous standard deviation formulas for calculating the televote–jury split gives us the following:

So what does that show? Well, it’s clearly an improvement: from the data we have so far, we can expect this latest voting system change to shift the effective televote–jury power balance from roughly 24-76 to 30-70. But that, quite clearly, remains far from the goal of an effective 50/50 split. If SVT still wants the televotes to matter, they’ll need to start thinking of some more changes – either to how the votes are combined or, more directly, to the app voting system itself. There are plenty of ways that either option could be done while keeping app engagement high, if that’s what SVT wants.

For anyone who might want to see more about how I reached these conclusions (or just pick holes in my reasoning), you can look at all of the raw data in my Excel file, or just leave me a comment – I’m always happy to somewhat-pointlessly overanalyse Eurovision stuff!

Some Stats on the Melfest App Vote: The Real Jury–Televote Split

UMK2016: Predicting via YouTube Stats

(Copy-pasted from my post on the ESCUnited forums).

Okay, so boredom just drove me to make some in-depth predictions for the national finals based on YouTube data. I went through all 18 songs uploaded on the official UMK channel, looking at the the view count, likes and dislikes, and comment count as my raw data, then did some basic calculations and threw the numbers together to rate a song’s chances of success based on five different areas. Those are:

  • Views: Quite simply, how many people have looked up the song as a result of pre-contest hype?
  • Likes: How many people have shown that they like the song, and how many supporters will it have going into the contest?
  • Approval: Out of everybody who rated the song, what percentage liked it?
  • Activity: What proportion of viewers enjoyed the song enough to take action and give it a like?
  • Comments: How much discussion, good or bad, is surrounding the entry in the pre-live show period?

All five of these, I think, look at separate and important areas for predicting a song’s popularity, and so I gave points from 1-18 to all songs in each area, then added these points to determine an overall prediction list, which is as follows (click image for larger version):

UMK2016 YouTube Predictions

Before I try to draw any other conclusions from it, I’ll note that there are some obvious errors in trying to predict things this way. In particular, this pre-contest YouTube audience may be quite different from the audience of Finnish televoters who will determine the result: it likely includes a lot of international views, a lot of ESC fans, and tends towards a younger demographic. The prediction chart doesn’t take into account any overlap between genres, or what may result from that; perhaps the more prominent genres here, ie. dance songs and power ballads, may split the votes within their genres, leaving fans of alternative music to focus their votes on a smaller number of songs. Finally, it’s impossible, of course, to take the staging potential or live performances of the entries into account until they’re performed, which may make a large difference.

However, we’re still left with a large data set on a range of areas for all of the songs, and I thought it’d be interesting to whip up this chart to see more concrete figures to make predictions with, rather than trying to base things on a few comments here and there. I don’t think the results are particularly surprising; the top four contenders score-wise seem to be those most widely-discussed online, while the lower-ranked entries are generally dismissed in predictions (with the exception of Lieminen’s entry, which has inexplicably short odds on some betting sites). Saara and Cristal seem to have the most attention on them from a wider audience at the moment, though the Approval and Activity columns show that Saari, plus Annica & Kimmo, are the best at gaining fans from those who do actually see them, which is arguably the more important quality in a national final setting, where all acts are viewed equally. Additionally, Saari’s large comment count suggests that either those who view the song react in some significant way, or that he has an ardent group of fans, either of which should help him to earn votes.

Setting aside the top four, I think the stats show three clear dark horses also in the running. Pää-Äijät have got a lot of views and likes relative to their overall position, indicating a controversial love-or-hate entry, which seemed to help the 2013 and 2015 UMK winners. On the other hand, Barbe-Q-Barbies haven’t received significant attention in terms of direct views, but are generally met with a high approval rating, and should stand out given the lack of other rock songs in the national final. Finally, Rafaela Truda’s video statistics are difficult to analyse, as most of her views and dislikes seem to have resulted from a plagiarism controversy of some kind. While the topic may be negative, it’s still given her a lot of exposure, popularised her song, and gotten people talking, which – with a catchy earworm of a song like hers – may conversely help her.

Given all of the above, I’ll make the following prediction for Top 7:

  1. Mikael Saari: “On It Goes”
  2. Saara Aalto: “No Fear”
  3. Annica Milán & Kimmo Blom: “Good Enough”
  4. Barbe-Q-Babies: “Let Me Out”
  5. Cristal Snow: “Love Is Blind”
  6. Rafaela Truda: “Rise Up”
  7. Pää-Äijät: “Shamppanjataivas”

I’ve never been great at predicting NFs, and I’m probably way off this time as well; I feel like my bias towards Saari may have made this chart a little too optimistic for me. Still, I enjoyed working on that for the last couple hours, and I’m curious to see if anybody agrees or disagrees. Any thoughts?

UMK2016: Predicting via YouTube Stats