Interpreting black box predictions using Fisher kernels. Google Scholar He, M. Narayanan, S. Gershman, B. Kim, and F. Doshi-Velez. The degree of influence of a single training sample z on all model parameters is calculated as: Where is the weight of sample z relative to other training samples. test images, the harmfulness is ordered by average harmfullness to the Understanding Black-box Predictions via Influence Functions International Conference on Machine Learning (ICML), 2017. Understanding Black-box Predictions via Influence Functions Proceedings of the 34th International Conference on Machine Learning . We show that even on non-convex and non-differentiable models We see how to approximate the second-order updates using conjugate gradient or Kronecker-factored approximations. The canonical example in machine learning is hyperparameter optimization. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. SVM , . Haoping Xu, Zhihuan Yu, and Jingcheng Niu. Understanding Black-box Predictions via Influence Functions ICML2017 3 (influence function) 4 Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. The In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. values s_test and grad_z for each training image are computed on the fly On the importance of initialization and momentum in deep learning. Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. Neural nets have achieved amazing results over the past decade in domains as broad as vision, speech, language understanding, medicine, robotics, and game playing. A. M. Saxe, J. L. McClelland, and S. Ganguli. Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. W. A theory of learning from different domains. A Dockerfile with these dependencies can be found here: https://hub.docker.com/r/pangwei/tf1.1/. The second mode is called calc_all_grad_then_test and To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. affecting everything else. The final report is due April 7. GitHub - kohpangwei/influence-release Using machine teaching to identify optimal training-set attacks on machine learners. . We are preparing your search results for download We will inform you here when the file is ready. Understanding Black-box Predictions via Inuence Functions 2. % In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. Y. LeCun, L. Bottou, G. B. Orr, and K.-R. Muller. The security of latent Dirichlet allocation. In. Springenberg, J. T., Dosovitskiy, A., Brox, T., and Riedmiller, M. Striving for simplicity: The all convolutional net. We'll consider two models of stochastic optimization which make vastly different predictions about convergence behavior: the noisy quadratic model, and the interpolation regime. In, Moosavi-Dezfooli, S., Fawzi, A., and Frossard, P. Deep-fool: a simple and accurate method to fool deep neural networks. can speed up the calculation significantly as no duplicate calculations take PDF Appendix: Understanding Black-box Predictions via Influence Functions grad_z on the other hand is only dependent on the training non-convex non-differentialble . On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks. We are given training points z 1;:::;z n, where z i= (x i;y i) 2 XY . This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. We'll see how to efficiently compute with them using Jacobian-vector products. , . For more details please see For a point z and parameters 2 , let L(z; ) be the loss, and let1 n P n i=1L(z on to the next image. Understanding Black-box Predictions via Influence Functions (2017) 1. Understanding Black-box Predictions via Influence Functions - Github PDF Understanding Black-box Predictions via Influence Functions Programming languages & software engineering, Programming languages and software engineering, Designing AI Systems with Steerable Long-Term Dynamics, Using platform models responsibly: Developer tools with human-AI partnership at the center, [ICSE'22] TOGA: A Neural Method for Test Oracle Generation, Characterizing and Predicting Engagement of Blind and Low-Vision People with an Audio-Based Navigation App [Pre-recorded CHI 2022 presentation], Provably correct, asymptotically efficient, higher-order reverse-mode automatic differentiation [video], Closing remarks: Empowering software developers and mathematicians with next-generation AI, Research talks: AI for software development, MDETR: Modulated Detection for End-to-End Multi-Modal Understanding, Introducing Retiarii: A deep learning exploratory-training framework on NNI, Platform for Situated Intelligence Workshop | Day 2. There are several neural net libraries built on top of JAX. This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: Understanding Black-box Predictions via Influence Functions --- Pang When testing for a single test image, you can then In contrast with TensorFlow and PyTorch, JAX has a clean NumPy-like interface which makes it easy to use things like directional derivatives, higher-order derivatives, and differentiating through an optimization procedure. Google Scholar Krizhevsky A, Sutskever I, Hinton GE, 2012. In. This We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. (a) What is the effect of the training loss and H 1 ^ terms in I up,loss? WhiteBox Part 2: Interpretable Machine Learning - TooTouch Uses cases Roadmap 2 Reviving an "old technique" from Robust statistics: Influence function Fortunately, influence functions give us an efficient approximation. Understanding Black-box Predictions via Influence Functions Deep learning via hessian-free optimization. The reference implementation can be found here: link. This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. Insights from a noisy quadratic model. This class is about developing the conceptual tools to understand what happens when a neural net trains. Understanding Black-box Predictions via Influence Functions - ResearchGate Limitations of the empirical Fisher approximation for natural gradient descent. Lectures will be delivered synchronously via Zoom, and recorded for asynchronous viewing by enrolled students. https://dl.acm.org/doi/10.5555/3305381.3305576. This isn't the sort of applied class that will give you a recipe for achieving state-of-the-art performance on ImageNet. Visual interpretability for deep learning: a survey | SpringerLink In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. We'll consider the heavy ball method and why the Nesterov Accelerated Gradient can further speed up convergence. Shrikumar, A., Greenside, P., Shcherbina, A., and Kundaje, A. Class will be held synchronously online every week, including lectures and occasionally tutorials. M. MacKay, P. Vicol, J. Lorraine, D. Duvenaud, and R. Grosse. Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. For toy functions and simple architectures (e.g. The answers boil down to an observation that neural net training seems to have two distinct phases: a small-batch, noise-dominated phase, and a large-batch, curvature-dominated one. PW Koh*, KS Ang*, H Teo*, PS Liang. Lage, E. Chen, J. On the Accuracy of Influence Functions for Measuring - ResearchGate Please try again. In Proceedings of the international conference on machine learning (ICML). All Holdings within the ACM Digital Library. The next figure shows the same but for a different model, DenseNet-100/12. ordered by helpfulness. In. Understanding Black-box Predictions via Influence Functions After all, the optimization landscape is nonconvex, highly nonlinear, and high-dimensional, so why are we able to train these networks? reading both values from disk and calculating the influence base on them. Understanding Black-box Predictions via Influence Functions We'll start off the class by analyzing a simple model for which the gradient descent dynamics can be determined exactly: linear regression. , mislabel . We'll mostly focus on minimax optimization, or zero-sum games. Understanding Black-box Predictions via Influence Functions J. Lucas, S. Sun, R. Zemel, and R. Grosse. Which algorithmic choices matter at which batch sizes? understanding model behavior, debugging models, detecting dataset errors, Check if you have access through your login credentials or your institution to get full access on this article. Influence functions help you to debug the results of your deep learning model This paper applies influence functions to ANNs taking advantage of the accessibility of their gradients. Most importantnly however, s_test is only However, in a lower Data-trained predictive models see widespread use, but for the most part they are used as black boxes which output a prediction or score. Jaeckel, L. A. Understanding black-box predictions via influence functions One would have expected this success to require overcoming significant obstacles that had been theorized to exist. With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. Pang Wei Koh and Percy Liang. ordered by harmfulness. No description, website, or topics provided. So far, we've assumed gradient descent optimization, but we can get faster convergence by considering more general dynamics, in particular momentum. Metrics give a local notion of distance on a manifold. Acknowledgements The authors of the conference paper 'Understanding Black-box Predictions via Influence Functions' Pang Wei Koh et al. We'll use the Hessian to diagnose slow convergence and interpret the dependence of a network's predictions on the training data. Helpful is a list of numbers, which are the IDs of the training data samples Then, it'll calculate all s_test values and save those to disk. the algorithm will then calculate the influence functions for all images by In, Metsis, V., Androutsopoulos, I., and Paliouras, G. Spam filtering with naive Bayes - which naive Bayes? CodaLab Worksheets [ICML] Understanding Black-box Predictions via Influence Functions Understanding Black-box Predictions via Influence Functions The idea is to compute the parameter change if z were upweighted by some small , giving us new parameters ^,z argmin(1 )1 nn i=1L(zi,)+L(z,). Your search export query has expired. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1885--1894. lage2019evaluationI. International Conference on Machine Learning (ICML), 2017. We'll consider the two most common techniques for bilevel optimization: implicit differentiation, and unrolling. Understanding Black-box Predictions via Influence Functions - SlideShare S. McCandish, J. Kaplan, D. Amodei, and the OpenAI Dota Team. Existing influence functions tackle this problem by using first-order approximations of the effect of removing a sample from the training set on model .