One of the great challenges of modern AI theory is to explain the success of overparameterized systems that learn to generalize even when optimizing over far more free parameters than examples. This success is often attributed to algorithmic traits such as inductive bias, implicit/explicit regularization, linear stability and more. 
A great test case, to study how such algorithmic traits allow learning, is the classical setting of stochastic convex optimization. Indeed, classical results already demonstrate how an algorithm can learn, even in the overparameterized regime, as long as the population loss is convex. Surprisingly, this is possible even when there doesn't seem to be any effective bound on the number of parameters in the model, or the Rademacher complexity of the class.  But, is that due to some type of implicit regularization? Flatness of the minima? Or even stability? What allows learning algorithms to succeed in the convex case? 
In this talk I will describe our recent analyses for the generalization of algorithms such as stochastic gradient descent, gradient descent, and, generally, first order methods. Through these works we can shed light on potential techniques to prove generalization in overparameterized settings, and revisit notions such as capacity, stability and regularization as well as their role in generalization.

Based on joint works with Idan Amir, Assaf Dauber, Meir Feder, Tomer Koren, Yishay Mansour, and Uri Sherman. 
 

Roi Livni is an assistant professor at Tel Aviv University EE. He received his Ph.D from the Center for Brain Sciences (ELSC) at The Hebrew University of Jerusaelm under the supervision of Amir Globerson. After that he was a research instructor at Princeton University, where he conducted his postdoctorate. His research focuses on Learning theory with special emphasis on generalization theory, privacy and generative learning. Roi is the recipient of several awards and fellowships. Such as the Google Phd Fellowship, Rothschild postdoctoral fellowship, COLT 2013 Best student paper award, ICML 2013 Best Paper award, FOCS 2020 Best paper award and COLT 2021 Best paper runner-up.

In the past decade, deep learning has completely revolutionized AI.  In this talk, I will explain what deep learning is, why it works, what it has done for us so far, and what it is likely to do in the future.

Ilya Sutskever is Co-founder and Chief Scientist of OpenAI, which aims to build artificial general intelligence that benefits all of humanity. He leads research at OpenAI and is one of the architects behind the GPT models.
Prior to OpenAI, Ilya was co-inventor of AlexNet and Sequence to Sequence Learning. He earned his B.Sc, M.Sc, and Ph.D in Computer Science from the University of Toronto.

Dror Bin is the CEO of the Israel Innovation Authority, an independent public entity that operates for the benefit of the Israeli innovation ecosystem and Israeli economy as a whole. Its role is to nurture and develop Israeli innovation resources, while creating and strengthening the infrastructure and framework needed to support the entire knowledge industry. Prior to his role at the Authority, Dror served as President and CEO of RAD Data Communications, a leading global telecom network solutions company with hundreds of employees at the company's headquarters in Tel Aviv, a manufacturing center in Jerusalem and development center in Beer Sheva as well as dozens of corporate branches around the world. Dror also served in a series of positions for close to a decade at Comverse Technology, the last one as a member of the management team and VP of Global Sales. Following this, Dror served as a venture partner at Carmel Ventures and a chairman at several of its portfolio companies. In addition, Dror served as a partner at Shaldor, a leading management consulting firm in Israel, where he led the development and implementation of business and marketing strategies for major organizations in the financial, consumer, retail, high-tech, banking and other industries. Dror holds two bachelor’s degrees from the Technion – Israel Institute of Technology: one in systems information engineering and the other in industrial management, as well as an MBA from Tel Aviv University.

Brigadier General Aviad Dagan, 49 years old, married and a father of four. In his most recent position, he served as a base commander for Hatzerim Air Force Base, Commander of the Northern Command Fire Center, and Head of the Air Force Participation Department. Brigadier General Aviad Dagan holds a bachelor's degree in computer science and law from Bar-Ilan University, a master's degree in law from Bar-Ilan University and another master's degree in national security from the University of National Security in Washington.

Isaac Ben-Israel was born in Israel (Tel-Aviv), 1949. He studied Mathematics, Physics and Philosophy at Tel-Aviv University, receiving his Ph.D. in 1988. He joined the Israel Air Force (IAF) after graduating high school (1967) and has served continuously up to his retirement (2002). During his service, Isaac Ben-Israel has held several posts in operations, intelligence and weapon development units of the IAF. He headed the IAF Operations Research Branch, Analysis and Assessment Division of IAF Intelligence, and was the Head of Military R&D in Israel Defence Forces and Ministry of Defence (1991-1997). In January 1998 he was promoted to Major General and appointed as Director of Defence R&D Directorate in IMOD. During his service he received twice the Israeli Defence Award.

After retirement from the IDF Isaac Ben Israel joined the University of Tel-Aviv as a professor and was the head of Curiel Centre for International Studies (2002-2004), the head of the Program for Security Studies (2004-2007), Executive Director of the Interdisciplinary Centre for Technological Analysis & Forecasting at Tel-Aviv University (ICTAF) (2010-2013), Deputy Director of the Hartog School of Government and Policy in Tel-Aviv University (2005-2015) and a member of Jaffe Centre for Strategic Studies (2002-2004). In 2002 he founded and headed the Yuval Ne’eman Workshop for Science, Technology and Security. He was a member of the Board of Trustees of Ariel University Centre (2009-2011), and a member of the advisory council of Neaman Institute for Advanced Studies in Science and Technology at the Technion (2000-2010). In 2002 he founded RAY-TOP (Technology Opportunities) Ltd, consulting governments and industries in technological and strategic issues.

Professor Ben-Israel was a member of the 17th Knesset (Israeli Parliament) between June 2007 and February 2009. During this period he was a member of the Security and Foreign Affairs Committee, the Finance Committee, the Science & Technology committee, the Chairman of the Homeland Security Sub Committee and the Chairman of the Israeli–Indian Parliamentary Friendship Association.

In 2011 he was appointed by the Prime Minister to lead a task force that formulated Israel national cyber policy. Following that he founded the National Cyber Headquarters in the PM Office. In 2014 he was appointed again by the PM to lead another task force which resulted in a government decision (February 2015) to set up a new National Cyber Authority. Isaac Ben Israel was a member of the board of directors of IAI (2000-2002), the board of the Israel Corp. (2004-2007) and the R&D advisory board of TEVA (2003-2007) and Chairman of the Technion Entrepreneurial Incubator (2007). He was the Chairman of Israel National R&D council between 2010-2016.

Professor Ben-Israel has written numerous papers on military and security issues. His book Dialogues on Science and Military Intelligence (1989) won the Itzhak-Sade Award for Military Literature. His book on The Philosophy of Military Intelligence had been published by the Broadcast University (1999) and has been translated into French (2004). His book Science, Technology and Security: From Soldiers in Combat up to Outer Space, was published in 2006. His book on Israel Defence Doctrine was published in 2013.
Isaac is married to Inbal (née Marcus) and they have three sons: Yuval (1981), Roy (1984) and Alon (1988).

Ziv heads Israel’s National Program for AI Infrastructure, which is a coordinated, collaborative government effort aimed at ensuring Israel’s future leadership in the global AI arena. The program is focused on the long-term infrastructure required for promoting sustained innovation and growth. At its first stage, the program focuses on four pillars – Establishing an Israeli High-Performance Computer; Creating infrastructure for Natural Language Processing in Sematic languages that will fuel innovation and support AI assimilation in public sector services; Extending human capital in the Israeli academia and removing regulatory barriers to AI based innovations. The program is a mutual effort of Israel’s Ministry of Innovations, Science and Technology, Israeli Innovation Authority, the Directorate of Defense Research and Development, the Higher Education Council and the Ministry of Finance. Ziv is a versatile multi-disciplinary technologist with profound knowledge of Artificial Intelligence, communication networks, big data, and distributed systems. He is an experienced manager with proven capabilities of working at matrix environments as well as direct management skills, a versed innovation leader, and a growth-oriented CTO. Ziv holds a MSc. degree from the department for Software and Information Systems Engineering at the Ben Gurion’s University of the Negev and is about to complete his PhD studies researching the vulnerability of AI systems to adversarial manipulaitons.

11:40 - 11:50

 Moderator: Prof. Dan Amiram, Vice Dean and the Joseph Safra Chaired Professor for Capital Markets and Financial Institutions, Coller School of Management, Tel Aviv University

11:50 - 12:10 - Contextual Multi-Armed Bandits meets Causal Inferencing

 Reuven Shnaps, Ph.D., Chief Analytics Officer, Earnix

Neural networks (NN) are often designed for pointwise predictions, but can also be designed to predict distributions. There are NN architectures specifically designed for distribution prediction (such as mixture-density networks), but we might have an architecture we designed for pointwise prediction, when we decide we need distribution prediction as well. In Meta, for example, we use NNs to predict the number of views content will get, but for reviewing potentially-harmful content, it is the upper-quantiles of the view distribution that turn out to be relevant.
 
In this short talk we will cover two "free lunch" methods for adapting existing NNs for distribution prediction: variational inference and MC dropout (the latter more commonly used for regularizing models). We will also cover two types of uncertainty, aleatoric and epistemic, roughly corresponding to randomness in the way the data was generated, and randomness in the way we algorithmically use the data. Finally, we will discuss the relationships between the two methods and the two types of uncertainty.
 

Ami is a research scientist at Meta’s Core Data Science team, and has been working with Novi (Meta Financial Services) over the past few years. He holds a PhD in electrical engineering from Tel Aviv University, and is the proud father of three girls (11, 8, 8) who are also in the field of data science (although they don’t know that yet).