{"id":82,"date":"2021-04-29T21:37:22","date_gmt":"2021-04-29T21:37:22","guid":{"rendered":"https:\/\/sites.ps.uci.edu\/clivar\/?p=82"},"modified":"2021-04-30T19:37:18","modified_gmt":"2021-04-30T19:37:18","slug":"idk-neural-networks","status":"publish","type":"post","link":"https:\/\/sites.ps.uci.edu\/clivar\/2021\/04\/29\/idk-neural-networks\/","title":{"rendered":"IDK: Neural networks that say \u201cI don\u2019t know\u201d to learn better"},"content":{"rendered":"\n

I teach the introductory atmospheric dynamics course for our masters students. A few years ago I noticed that the incoming students had perfected the art of partial credit – finding ways to get as many points as possible on exams without actually knowing the answers. While this may have led to a better final grade, it sort of missed the point. Furthermore, a critical skill for any scientist is to “know when they know and know when they don\u2019t know”, and this class seemed like a great opportunity to try and get this message across.<\/p>\n\n\n\n

A critical skill for any scientist is to know when they know and know when they don\u2019t know<\/em><\/p><\/blockquote><\/figure>\n\n\n\n

Because of this, the past few years I have implemented a new grading system on exams, where partial credit is no more, but students are allowed to write \u201cIDK\u201d (I don\u2019t know) on any question and turn their solution in 24 hours later for half credit. The idea is to make them pause and assess whether they actually know the material or not.<\/p>\n\n\n\n

An unexpected outcome of this grading exercise was that it made me reconsider how I use artificial neural networks (ANNs) in my scientific research. Why do I value scientists knowing when they don\u2019t know, but turn around and train an ANN to provide an answer for everything all of the time? Put another way, I would prefer an ANN say \u201cIDK\u201d rather than get it horrendously wrong.<\/p>\n\n\n\n

The concept of ANN confidence is nothing new, and in fact, uncertainty quantification for ANNs (both of the inputs and outputs) has become a very hot topic as of late. Furthermore, ANNs designed for classification tasks, to a certain extent, already output confidence via their softmax scoring. So, what\u2019s the novel thing here?<\/p>\n\n\n\n

Well, in many situations, we would never expect – in fact, it may be impossible – to get the answer right all of the time. Instead, we must look for particular states of the system that are more predictable than others (i.e. windows of opportunity<\/strong>; see Mariotti et al. 2019<\/a> for applications to weather prediction), and do our best to learn from and exploit these periods of predictability when they occur. By training ANNs to provide an answer for every prediction, it is likely that they are wasting energy trying to learn samples that may not be predictable in the first place.<\/p>\n\n\n\n

\n
\n
\"\"<\/figure>
\n

Take the simple 1D example in the figure to the right. Naively fitting a straight line through all of this data would result in a fit that performs poorly on most samples. Instead, I want an ANN that can learn to predict the samples along the line with high accuracy and also identify the samples within the point-cloud as being highly unpredictable (say \u201cIDK\u201d). That is, I want an ANN that can say \u201cIDK\u201d while it is training<\/strong>.<\/p>\n<\/div><\/div>\n<\/div>\n<\/div>\n\n\n\n

Motivated by this, the past few months my collaborator Dr. Randal Barnes (yes, there’s a relation…he’s my dad) and I explored ANN loss functions for regression<\/a> and classification<\/a> tasks, which we have termed Controlled Abstention Networks (CAN). The CAN identifies and learns from the more confident samples and abstains (says \u201cIDK\u201d) on the less confident samples during training<\/strong>. Our work leans heavily on that by Thulasidasan et al. (2019)<\/a> and Thulasidasan (2020)<\/a> which first introduced us to the concept of a simple classification loss function for abstention. What we\u2019ve found is that when the network is able to abstain<\/strong> <\/em>during training<\/strong>, it learns the predictable relationships better. <\/p>\n\n\n\n

…when the network is able to abstain during training<\/em>, it learns the predictable relationships better.<\/p><\/blockquote><\/figure>\n\n\n\n

\"\"<\/figure>
\n

Returning to our 1D example, the CAN is able to identify, and learn from, the samples that fall along the well-defined line. Furthermore, the CAN learns to abstain on the samples within the point-cloud. Certainly this example is incredibly simplistic, but in our papers, we go beyond this 1D example and demonstrate the utility of the CAN for multiple complex climate use cases. <\/p>\n<\/div><\/div>\n\n\n\n

What gets me most excited about the CAN concept is that it spans a wide range of applications. We have found it useful for discovering particular input states that lead to predictable behaviour, identifying labels\/outputs that are more predictable than others, or even acting as a data cleaner (i.e. filtering out noisy samples) during training to learn the non-noisy samples better. <\/p>\n\n\n\n

I don\u2019t ever expect to be able to predict the dynamics of the climate system perfectly – chaos theory sort of put that one to bed. Instead, abstention during training can help me search the data for predictable relationships, accommodating “IDK” along the way. That\u2019s a good thing, as I doubt the Earth system gives partial credit.<\/p>\n\n\n\n

\n\n\n\n
References for the CAN<\/h5>\n\n\n\n