For more infomation >> How to make a presentation? Effective presentation - Duration: 12:16. 
For more infomation >> Trump Clears The White House In Staggering Move To Make Way For Special Guests He Wants To Pray With - Duration: 4:09. 
For more infomation >> Autopilot Money Making System 2018 - How to Make Money Online - Duration: 3:41. 
For more infomation >> 4 Ways to make a man feel special - Duration: 1:19. 
For more infomation >> Bees struggle to make it through tough spring - Duration: 1:16. 
For more infomation >> Malloy set to make major jobs announcement - Duration: 0:14. 
For more infomation >> Тайны Коко-Как резать бумагу по-мексикански/Субтитры/Coco-How To Make Papel Picado - Duration: 2:20. 
For more infomation >> District attorney said he never asked to make changes to police report - Duration: 1:30. 
For more infomation >> CCSU to make changes to sexual assault reporting - Duration: 2:42. 
>> I'm very happy to have Bo Waggoner here today.
He's currently a postdoc at Penn.
Before that, he got his PhD at Harvard.
He went to undergrad at Duke.
He's actually been interned at MSR in New England,
and interned twice at Google.
He is also a really serious runner.
He even won ACC award for
Athlete of the Year.
So, let's hear what he wants to talk to you about.
>> Yeah. Thanks very much. Thanks for
coming to the talk and putting this together,
especially thanks to Alex for making it happen.
Yeah, I'm really excited to get to talk to you guys.
So, the title of the talk is,
Acquiring and Aggregating
Information in Societal Context.
So, I want to think about a paradigm
that's very familiar to many of us,
especially in Machine Learning.
There's a bunch of information,
we feed it into our algorithms and they
output predictions or decisions.
So, it might be data about the weather,
and the output might be some classifier that
classifies weather as indicating sunny,
or rainy in the next day or whatever.
But while we traditionally like to
analyze our algorithms with this perspective,
actually, there's this recognition that
they live in a context that's bigger than this.
So, context of people.
In particular, the data that's coming
in is data that comes from people.
The data maybe about people in some way,
and the outputs, the decisions,
or predictions, also are affecting people.
So, this raises some really interesting questions.
I think both from a societal standpoint,
how should this be designed when we're
taking into account this context?
There are often really nice technical questions,
which I really like, coming
from more of a theory background.
So first, I just want to tell you briefly
about the kinds of questions that come up,
and then which ones I'll be
talking about today in more detail.
So, I like to
break the challenges down into three rough categories.
So, one is strategic behavior of agents.
So, when you take something like
machine learning type system,
and put it in the context with people,
they may be behaving strategically.
So, this input data you're getting,
for example, might not be the real or true data.
As one example of how they might behave strategically,
they could just misreport data.
They might do that because they care about
the outcomes, for example.
Another big concern is privacy of this data.
So, learning algorithms traditionally
weren't designed with any sort of privacy in mind,
but we often want to now redesign them,
or add this in because people care
about this data may be sensitive or private in some way.
This is a whole field of differential privacy,
and I'll talk a bit about that.
A final aspect, but one I won't get to today,
is this idea of fairness for decisions.
So, the decisions that
the algorithms make affect people in certain ways,
and the traditional approach of just trying to get
high accuracy may have ethical problems,
when they're put in that context of people.
So, it's a really interesting area.
There's this whole idea of fairness in Machine Learning,
but I won't get to that today.
So, instead I'll be talking about
projects from these first two categories.
Okay. So, the outline for the talk.
I'm going to tell you about a project dealing with
strategic behavior in Machine Learning.
So, how to do some learning in a situation where
agents are manipulating the data.
We'll spend more time on that,
and we'll spend a little time talking about
challenges that come up when we try to do
private Machine Learning and a project about sort of
translating theory
of private Machine Learning into practice,
and some challenges that arise,
then some stuff on future directions.
So, that's the outline. Please definitely stop me,
questions, discussion, would all be great.
I'd rather have some really interesting discussions here,
and have a little less time here,
if there are interesting questions to talk about.
Okay, good.
So, here's my overall picture of learning algorithms,
or algorithms in general in societal contexts.
For this first project, I want to focus
on the strategic behavior aspect.
So, this question of how do we do learning when
these people are being strategic about
the kind of data they give to the algorithms.
So, we're going to look at a particular setting,
and I want to motivate it with an application,
an example of detecting credit card fraud.
So, a high level example,
but imagine that over a series of rounds,
agents make a transaction,
and the system sees some data about this transaction,
and it wants to know if that's a fraudulent transaction,
or an honest one.
You can think of maybe
some feature vector is describing the transaction.
So, maybe it is a vector of real numbers or something.
So, the system is going to classify
it as being honest or fraudulent,
and has some classifier,
I'll be using beta for this.
Eventually, the system will learn what
the true status of that transaction is, let's say.
In fact, let's just
simplify things down and assume it learns very quickly,
like immediately, if it was an honest transaction,
or if it was fraud, because it gets
feedback from the account holder.
Then it wants to
update and learn better
classifiers based on this feedback.
So, what's the big challenge?
So, the challenges is that the fraudulent agents in
these systems are going to be
responding to the classifier that you pick.
So, that's why I put this hat on top of XT,
to denote that this future vector we're seeing,
or this transaction we're seeing,
may not be the truth that
these fraudulent agents would
like to have purchased, for example.
So, if there weren't some spam feature,
they just purchased whatever their favorite thing is.
But they're going to be- Instead of spam,
I should say fraud detection, right?
If there weren't fraud detection,
they just purchased whatever they want,
but they're going to be responding.
So, depending on what
the algorithms classifier looks like,
they're going to be buying things in such a way
to try to evade the classifier.
So, they're going to choose
these transactions strategically
depending on what classifier
you're deploying in the first place.
So, here's a picture of what that might look like.
So, if you picture each of
these transactions as a point in space,
we're modeling it as a point in R or d,
and you may have
all these blue points representing honest transactions,
and these red points,
I want you to think of these as the transactions
the fraudster would make if there were no consequences.
So, they'd like to buy, let's
say like chocolate ice cream,
or they'd like to buy a boat, or whatever,
but they're not going to
necessarily get away with that
because there's a fraud detection scheme.
In particular, if we had all this data,
we could deploy some classifier.
It might not be perfect,
but it would classify honest transactions from fraud,
and it would catch this fraud.
Yeah, there are obviously some mistakes.
So, I had a fun incident,
where my credit card was declined making a purchase,
and the location was the city I was living in,
and the purchase was running shoes.
So, I really don't know- if you know me,
that's like most of what I spend my income on.
So, I don't know what was-
These things aren't perfect in practice.
But life is even harder
because if the fraudsters
know that this is the classifier you're deploying,
and let's say in the worst case they
know or they have some good idea,
they're going to be responding
and modifying their transactions.
Okay. So, what you're going to see,
is going to end up looking more like that.
So, maybe instead of chocolate ice cream,
you have to buy strawberry,
maybe instead of a boat, you have to buy
a kayak, or whatever.
But you modify you transaction,
some way they try to increase
your chance of getting past this filter.
So, the filter has to be designed,
this detection classifier has
to be designed with these reactions in mind.
So, this will lead to some,
in game theory, the idea of the equilibrium.
But even knowing that fraudsters are best responding,
you want your classifier to be the optimal thing.
So, this is the perspective people have been taking.
So, there are a couple of papers on this problem.
Let me tell you a bit about the approach that they take.
So, at a high level they'll say,
suppose we have a bunch of data
drawn maybe IID from these populations,
some of it is honest, some of it is
fraud, it's all labeled.
Usually, these papers will suppose they
know the preferences of the fraudsters.
So, for example, one assumption might be
that there's a vector or direction,
and the fraudsters all want to
modify that direction or something like this.
So, using that information,
they can compute this classifier that does
well even given that they're best responding,
the fraudsters are best responding.
So, given that, learn some hypothesis,
some classifier, and it should do well.
So, the work I'd like to tell you about,
is going to try to weaken some of these assumptions.
So, one thing we want to think about
is an idea of agents arriving online,
and without having necessarily a distribution.
So, if you're familiar in Machine Learning,
often there are these two paradigms.
There's a IID data,
and then predicts well on that distribution.
But we instead want to think of
this worst case possibilities.
So, some transaction just arrives.
It's not necessarily from a distribution,
but in the long run we'd like to perform well.
So, after you sum notion of
regret, which I'll tell you about.
Another one is, we
really want to think about these agents,
especially the fraudulent agents.
It's heterogeneous and different.
So, they don't necessarily all want the same thing.
One might want ice cream,
one might want to sail across the river, or whatever.
A final important point is
that we didn't want to assume that
you can see the preferences of these fraudulent agents.
So, we don't necessarily know what they want.
We just see the transactions they tried to make.
So, we maybe saw them buying strawberry ice cream.
We'd like to try to infer from that,
did they actually want chocolate,
but they've modified it because of our classifier?
Or like do they actually like strawberry?
How can we use that to design better classifiers?
I hope that point makes sense.
Okay. So, that's the high level goal.
So, the paper is called strategic classification from
revealed preferences with some wonderful coauthors
at Penn or formerly of Penn.
Okay, and there are
three aspects to it that I'd like to discuss with you.
So,the first is how do we
model these aspects that I just talked about.
So, we have this idea of online arrivals and
limited feedback or limited knowledge about the agents.
And then what should we be shooting for?
And I think there's an interesting question there,
how do we measure how well we're doing?
And I'll explain why it's not super clear.
And then finally, how do we actually
try to solve the problem?
And for this talk with you today,
I actually want to,
I just want to spend a lot of time on
these because I think they're really interesting
and just a little bit of time on our solution
relative to maybe a normal outline.
So, it'll turn out that we can reduce
to known algorithms or
at least leverage them very heavily.
But we put a lot of
thinking and effort into this modeling questions and
I think that's also really
interesting for where to take future directions.
So, question so far?
I'll define the model next. Okay, awesome.
Okay, so in the fraud example I gave you before,
you already saw basically the model.
We'll make it a little bit more precise.
So, each agent that arrives,
they're either a fraudster or
an honest agent and they have
some what I'm saying true features,
which is the transaction they really want to make.
That's what we're thinking of it as.
Okay? And they have some true label.
Which is let's say one if they're
honest and minus one if they're not.
And finally, we assume they have some utility function,
which is measuring what's their utility
for facing this classifier beta or
this system described by beta when
their true purchase that they like to make is
x and they make a purchase x hat.
There are features that they present
to the classifier actually x hat,
so they've modified it in some way.
Okay.
So, what is this trying to capture?
Well, the utility intuitively has some tradeoff,
where if they're getting classified as
fraud or spam for certain,
then they're not going to get much utility.
But on the other hand, if they make
some transaction that they didn't
want in the first place and it goes through,
that's not all that useful to them either,
like strawberry ice cream, right?
So, apologies to anyone who actually likes,
I think I'm safe.
Okay. Good. So, we're
going to assume for this work that
honest agents don't act strategically in this way.
They just make the transaction they want to make.
So, they always said this x hat t.
This transaction actually ends up
happening just equal to the one they wanted.
They're not thinking about fraud detection schemes.
But we're assuming these fraudsters
do act strategically. Yes.
>> So are the fraudsters are being myopic or not [inaudible]
>> Yes a great question.
So, we assume they act myopically.
Yes.
So, we're going to assume that they
best respond with this utility function.
Yes.
>> Are we going to assume with their IIDs?
>> Yes.
So we're not going to assume they're IID for this.
Yes.
I mean I guess it would cover
that case and you could do some result for that.
Yes. Good yes. So we do assume
that the fraudsters will choose
this x hat t myopically
to maximize this utility function.
Yes.
Good. Okay. So we have this online model.
So there's a sequence of rounds.
The system has a current classifier any time,
the agent at that time
arrives and we assume the system observes okay
this transaction that they made or tried to
make and then it will learn this true label.
So for example if it was a fraudulent transaction,
even if they let it through,
the person who owned the credit card
will let them know that
it's being used wrong and they'll flag it as fraud.
Okay, so the system gets to learn
the true label for all the transactions.
Is that clear? It's okay.
Okay. And then we're going to use
some loss function to
measure how well the system performed on this data point.
Okay. So it's a loss as a function of
this classifier that you
were deploying or this hypothesis.
And then the transaction that was made and the label.
Okay. So this is just some function measuring performance
and natural one you might think of is that
this classifier is taking in X hat and just
saying one or minus one honest or fraud.
Okay. And then the loss could
be a loss of zero if it got it correct.
Right. So if it labelled it correctly according to
IT and a loss of one if it got it incorrect.
Okay so that's a natural loss function.
We won't end up working with it because
it's not computationally very
tractable but it's a good one
to keep in mind for the model.
And by the way so this model that I'm
presenting to you just as a heads up is very general.
We're going to end up solving some special case with
some strong assumptions but I think
it's fun to think about this general case so that in
future work people can try to solve more aspects of it.
Okay good. So the data points arrived,
the system had some loss,
the updates to a new classifier,
the next data point arrives.
Okay. Any questions about the model?
Yes.
>> [inaudible] with credit
cards [inaudible] particular person that
can be used in purchasing and the like [inaudible]
all the non- fraudulent?
>> Yes. That seems fair. One-
>> [inaudible].
>> Yes. So the formalism
that at this level I'm talking about can almost
even capture that because this xt
could include the cardholder's history,
somehow it can include their location or other things.
So it could probably include all that in this formalism.
Yes. In practice you might suppose you want to
just separate these problems like for each person.
>> It's like you give up the data points including like
features of the person
and what they're trying to purchase.
>> Yes, because we know
this credit card is associated
to this person or something.
So the other application we're thinking about as
motivation which you may have heard from
my verbiage is spam detection of emails.
So, honest people just
send the emails they want to send and
spammers would like to send
some particular spam email but it will get filtered out.
And so they try to modify it in some way and
maybe it's less effective but it gets
through the spam filter or something.
Yes. And again there you can think about
personalization versus everyone's data together.
>> [inaudible]
>> Yes, great question. Yes. So, does
the fraudster know the exact classifier that we're using?
So, in principle, there's no reason they would,
like if you're thinking about
credit card detection or spam filtering.
There's no real reason that we would publish,
for everyone to see, the exact algorithm
we're using at the current time.
But we're thinking of
that a little bit as security through
obscurity and that that's not
a good defense to assume
that they don't know the classifier.
You might be worried that,
if you think about like spam emails or something,
they send a ton of spam,
they can quickly start learning what
your classifier's doing at the current time.
So, this is a sort of worst
case assumption where they just
know the exact classifier and does respond to it.
So, we're thinking of it as
a worst case security assumption.
Question?
Okay, cool.
So now, I want to talk about measuring performance,
and why it's not obvious how to do that,
but how regret can be used.
So, the key ideas is that we're assuming that
these fraudulent agents are best responding, okay?
So, if they're honest, as I said,
they just make the transaction they want, but if not,
so let's define this function x hat t of
beta to be the best response.
So, the thing that the agent actually does,
the transaction they actually make, right?
So for the honest agent,
they just make the one they wanted,
but the dishonest agent is
doing some utility maximization.
So first, let's just define our performance.
So, this would be something pretty familiar and
normal in these machine learning settings.
So, we're going to take an average over all time,
capital t time steps,
of the loss that we experienced at that time step,
which is measuring how well did our classifier beta t do,
when the agent played x hat of
t of beta t. They responded to it,
and this was the true label.
Okay, so that's fine.
But normally, in this kind of model,
you look at regret as
compared to the best thing
you could have done in hindsight.
And now, there's a small wrinkle.
So, it's not really fair to
look at all the fraudulent transactions that were made,
and put them as dots like I did and then just say, "Oh,
If we had drawn this classifier,
we would have classified them all correctly."
So that's not fair because if
we had deployed this classifier,
they would have been modifying
their locations to respond to it.
So we can't just look at the past data and say,
something that did well in the past data,
would have actually done well if
deployed because if deployed,
the agents would have responded.
Okay. And so, this leads to this idea
that we want to compare to
the best fixed classifier in hindsight,
as we often do in these learning settings
but if we had used a different one,
they would have responded differently.
So, we need to compare to how it would have
performed had the agents responded to it.
What's the best thing in hindsight
knowing the agents would respond.
Okay. So, this was our performance
and then this is what OPT should look like.
The key point of this discussion is
that if we have deployed beta star the whole time,
we would have experienced loss of
beta star on the agent's responses to beta star.
So, this looks a little bit different
than the standard online learning where you might say,
"Take all of these x-y pairs and
plug in the performance of
beta star on those same x-y pairs.
This is taking into account that the
x's would actually look different.
The difference between these two performances
is just called the average Regret.
In the paper, we use this term, Stackelberg Regret.
The term comes from Stackelberg games in game theory
where a leader commits to
an action and the follower best responds.
So in this case, the follower is
best responding to the classifier.
So this notion of regret,
you can actually find in
previous papers in other literatures, though.
So things like security games and probably
if you're looking at online pricing
and auctions or something,
they would be using this kind of notion.
Although, we haven't seen it given a name
but we think Stackelberg Regret is a good name.
>> Is it still [inaudible] partial monitoring because I think the
enterprise literature on [inaudible].
>> Okay.
>> Like they have generalisation matrix.
>> Okay. Yes. So, it
is natural if you're thinking about this problem.
So, it's not surprising that people
have defined it in different places.
Good. But an interesting point about it is
that you can't- Yes, go ahead. Sorry.
>> So then the y's are also different, right?
>> No. So, that's important.
So, think of each agent that arrives, each x-y pair,
they're either a fraudster or they're
honest but they change.
If they're fraudster, they change their x.
>> They change their transaction.
>> They change their transaction x but their y is still,
were they honest were they not.
Yes, and one thing
that's interesting about our model
is that it's really baked in,
like their label is whether or not they're strategic.
So, that's an interesting aspect.
>> Do you need any assumptions on what fraction is fraud?
>> Good question.
Yes. So, we don't
need assumptions on what fraction are fraud versus not.
One thing we tried to do in
an extension is perform better
if there's a small amount of
fraud and you'll see briefly why.
So, a key point that's interesting
is if you just have the past data in front of you,
you can't really compute this OPT.
You have to know what these agent's utility functions
were in order to compute how they
would have responded if you had done something different.
You don't actually know that. You don't
know what their utility functions were,
you just got to see what they actually did on that day.
But we'll show how to compete with it,
how we can have an algorithm
whose performance competes with OPT,
or at least under some
assumptions under which you can do that.
I guess, I wouldn't be
surprised if everyone knows what average Regret is.
But just to be sure,
the point is that if average regret is going to zero,
then your loss per day is approaching that of OPT.
So that's good. So, you want
this average regret to go to zero.
If it goes to zero very fast, that's better.
By fast, I mean with t. So we're
often interested in average Regret as a function of time.
Okay. All good?
Okay. So now, let me tell you.
So, I tried to set up this relatively general question.
Let me tell you this specific setting
that we're able to solve for this paper.
So, we're assuming some linear predictor.
So, beta t is also
a vector in r to the d. The prediction,
you can think of as this number, beta.x.
Okay, so you can think of
positive means honest and negative means dishonest.
Very positive means I'm very
confident that this agent is honest.
Very negative, very confident they're dishonest.
So, we think about
logistic loss and we can also do hinge loss,
but we just focus on those two loss functions.
Okay. So this, I guess,
most people have seen this before but again,
this beta.x, this number,
if it's positive it thinks they're honest.
So if it matches y, if the sign of this is y,
then you have low loss.
If the sign doesn't match y,
so you predicted positive but they're actually fraud,
then you have a high loss, roughly.
But it's sort of a soft version
of zero one or it's trying to
be some soft version of zero one. All good.
Okay. So now, a very key assumption that we make,
what does the utility look like of these fraudsters?
What are they trying to do?
Okay, we assume that their utility of the form beta.x.
So, being classified as more positive is better for them.
So, the more honest
that they're perceived to be, the better.
But they pay some cost.
So notice this is beta.x-hat.
So, this is what the classifier predicts for
them but they suffer
some costs for having to modify from x to xn.
So, they have to trade off these two things.
We make some assumption on the class of
distance functions that are
measuring the cost for making this modification.
>> [inaudible]
>> Yes, good. So yes.
The reason is that we want to allow
heterogeneity among time steps.
So, an agent today
may care about different things than the agent tomorrow.
This is measuring that to say that,
like the agent today doesn't really mind switching
between chocolate and strawberry,
but the agent tomorrow minds very much.
So they have different distance functions.
>> What is the rationale of putting
this kind of disconnect when
you are putting it through the link,
one link for one agent but the other agent?
>> Do you mean,why one, the honest agent?
>> Looks linear.
>> Sorry so.
>> So, like the other loss
has this kind of hinge like form.
>> This one.
>> Right, whereas the process utility
is just fully of linear.
>> Yes. I see. I see.
So, maybe it wouldn't make
sense for their utility to be like the negative
of our loss or something like the worse we do the
happier they are or something.
>> [inaudible].
>>Yes, so it'll turn out that this is very
important for our whole approach, this particular form.
Yes, it would be
very interesting to be
able to solve something else like that.
I think one thing to mention though is,
one thing that keeps us really interesting is that it's
not a zero-sum game,
which your proposal would also not
be zero-sum because of this, I guess.
If it's a zero-sum game,
I think things are just less interesting because, yeah.
>> Is it realistic to assume this a utility function?
>> That's a great question, yes.
>> So one reason is to say like okay this
the best we can do technically.
>> Yeah.
>> Now does this merge anything.
>> Yeah.
Yeah, so we don't really know.
We don't have a particular scenario or like
we looked at these agents and they
definitely have this utility.
We can try to understand when this would be reasonable.
We don't think the answer is never.
So we think there are some scenarios.
So the most reasonable thing.
The first thing you would think of
probably could be like a 1 0.
So like my utility is one
if I got off my transaction went through.
So I was incorrectly classified and 0 if it didn't.
That would be maybe the most natural positive thing here.
But yeah go ahead.
>> [inaudible].
>> Well, yes.
So one thing we were thinking about is if
the classifiers using some randomisation
that's proportional to this.
So I'll allow things to go
through maybe with probability proportional
to the certainty of a classifier
then that this actually does make sense as
a utility could be.
But yeah, the main if you'd
go for something else from this
we don't know technically how to solve it.
So it would be great to do that.
>> I just wanted to mention that the 0 1,
I'm not necessarily sure that that's
the most natural thing because the thing where you
said before I guess it's unreasonable to assume
that attackers have a precise knowledge of data, right?
Here is also reflecting
that like the attackers want to make
sure be confident about
transactions will be classified as-
>> Yeah, thanks. That's something I forgot to
mention but is a great justification.
So yeah if you don't know
but you know that the higher this is the more
likely you are to be to pass through the filter then you
just try to maximise this that's some justification.
>> I guess it's not obvious to me how this
captures just like the inherent value of x or x half.
If x is like, I don't know,
like a 51-foot yacht and x half is a 52-foot yacht.
I don't really care.
Like they both have incredibly high value.
>> Yeah.
Yeah. So in this project
we're at a very high level of abstraction
and it's not clear what
the space are to
the deep space these things live in mean.
>> But is there a way, I mean
like some x hat are just going to have
intrinsically higher value or
some xs are just going to
have intrinsically higher value.
It seems like the only place that can
be captured in the prediction part.
>> Yeah. Yeah. So or maybe another,
I was going to say maybe you
could just have a coefficient in front that's
like how much I like
getting it through but you're right because I get
different utility for
different transactions going through.
Yeah. So yeah that may
that might not be captured by
this. That's really interesting.
>> Couldn't the distance
metric also include dollar amount?
>> Yeah. So yeah we can at least make these comparable.
Yeah. So you could sort of say,
yeah actually maybe this coefficient can
depend only on my true xt.
And then the distance is saying like you know
my distance from 51-foot yacht had
the value of 51-foot yacht minus
something number of feet.
Yeah. So yeah, and I'm glad we're having
the discussion because this is
a great direction for future work
can you what else can you
do besides these assumptions we're
making is really interesting.
Okay, so and our approach isn't going to work for
all Dt but for
some relatively large set of especially norms
of the of x minus x set and so this is
a pretty general almost completely general example
of the kinds of distance functions we can have.
So maybe have some matrix A and you take Ax and you
take Ax hat and you look at the distance
between them in some p norm
and you raise that to a power.
It's actually important for us that
this power be strictly greater than one.
The reason for that is actually intuitive.
There's this idea of a convexity involved.
And convexity is capturing diminishing returns.
So if I move this
a certain distance away and I
move that same distance again,
it's even more costly
and we actually need that to get results.
Otherwise you could have some,
this might not have a maximum.
You might be able to get
arbitrarily high utility by
moving farther and farther away arbitrarily.
Okay, good. So the main result
is under these assumptions
and in this setting that I told
you about we can reduce to
some online convex optimization algorithms
and use that to guarantee some regret.
So let me just really briefly tell you
that result and the idea of it.
So think of this loss at time
t as just a function of the classifier that
you chose to deploy at time t.
So that you get that
by plugging in beta in several spots here.
But think of it as one big function of beta.
Okay.
Well, if we can get that function to
be convex as a function of beta,
then we can apply things we know how to do.
Convex optimization.
Okay, and so that's what we show is true and
main tools using some convex analysis
which details which I
don't think are important for this talk,
so I'll skip it.
But so the corollary is
by applying various convex optimization algorithms,
you can get regret that's going to zero
as t increases some very fancy one you can
even get I think one over root t or one over t to
the two-thirds with better dependence on dimension.
Okay and this is despite I should emphases not
knowing exactly what this lost function
is that arrive right.
So we don't know this whole,
wo we have now a sequence of loss functions at each time.
They're each different.
And we don't know what
that entire loss function is
because it depends on their utility.
We just know that it's convex.
And there are convex optimization algorithms
that actually work in that setting.
>> [inaudible].
>> Yeah, I think so.
Yeah. Yeah. Good. Yeah, and so, sure.
Yes. So just quick extensions and future work.
Okay, so an extension
is you can treat these honest updates and
fraudulent updates differently and
do a better job with your algorithm.
The reason is that when an honest agent
arrives you actually do
know this convict's last function,
because it's just your last function.
If you change beta the thing
the data they gave you is not going to change.
So you can computer gradient
and you can do it fast a good update stuff.
When the fraud fraudulent update occurs,
you can't do that and you have to do something that
doesn't use gradient information.
If for those of you who are familiar
with such algorithms if not it's not a big deal.
Okay, so we have somewhat more general agent utilities
and also hinge lost.
The ideas are the same
but there's tons of room for future work.
I think it's really interesting as we've been discussing.
So other loss functions definitely.
Other forms of agents utility
definitely really interesting and especially if,
so you know our whole general approach
here once we formulated the problem ended up being
to reduce to this convex optimization which we
know how to do in this online setting.
So I mean one way to push on this is to just
say well here's another case
where we can use convex optimization.
Here's another case to go outside of that I really
don't know what approach you can
take that would be really interesting as well.
Okay. Yeah, go ahead.
>> So I guess one thing that makes me slightly nervous [inaudible]
optimization is that it
actually becomes really crucial that
when the attacker is in fact best responding to
the real knowledge of your data, right.
And so if they were working
with some sort of imperfect knowledge
about what I'm doing or
if I just like one very common thing
that goes on in these algorithms is you
have your current set of parameters and
because you can't observe a gradient
you randomly perturb it.
So there is no way in hell they
can know what random [inaudible] I actually applied.
>> Yeah.
Yeah, no, that's a great point.
Yeah. I think that's a disconnect between
our model and how you would deploy this in reality.
So maybe for the recording
I should say so the question was,
this is really relying on agents exactly best
responding to the exact beta we deploy.
And you're correct, we're using
these convex optimization algorithms that perturb
data a little and see how the gradient or see
what happens and use that to guess the gradient.
So if agents aren't best responding to the exact data
including this perturbation then
it's not clear this is going to work.
Yeah, I think another,
maybe there can be some way to get
around that by if you make
some assumptions about agent utilities,
you could guess that maybe
the gradient like locally you can guess
that they won't change too much and
then just use the gradient of your own loss.
So yeah, it should be addressable I would hope,
but not in this paper.
It's a great question.
>> Anything else, yeah.
>> I was just trying to think as at
a current price or whatever my utility would be.
It seems like it would be like a lot related to
the dollar amount of the item or
the dollar amount you could get from selling it.
So probably some things are easier to sell than others.
>> Yeah.
>> So, maybe if you could run like, if it all comes out,
suppose it all comes down to one number like
the dollar amount and you can sell something for,
will that simplify it what you do with it?
>> Yeah. Maybe, yeah. I think this is really
similar to the previous question about,
could we sort of this utility
doesn't seem to directly depend on something
like dollar value amount but could we
somehow argue that that's
a special case where there is
a coefficient which is the dollar amount of x.
>> [inaudible] second terms, you could just put them in dollars.
>> Yeah. So it seems to me possible.
Well, okay.
So we wanted A to be invertible but if you think of maybe
mapping these purchases into dollar amounts, okay?
If it's mapping down to
one dimension maybe it's there's a problem with
our formalism but it seems possible.
But yeah, when I don't want to claim this till
he is capturing too much because it's
a pretty special form so I'm not sure.
Good. Okay great. So yeah.
So, while I have you I'd like to give you
a different example of this machine
learning in societal context which is going to be
privacy focused example and much briefer of course.
Yeah. So, strategic behavior
was one really interesting aspect of this idea.
Another one that I'm interested in is this idea
of, maybe the agents,
they don't have a particular concern
about what this classifier is
doing so it's not like you know it's
classifying their own transactions.
But there is this privacy concern that it's going to
reveal information about my data
and differential privacy is a field which turns out to
be some very nice theory
and also moving into practice for addressing this issue.
So, I think I can probably guess that
everyone has seen differential privacy
a few times or many times.
So, I hope you don't mind, I think I'll
skip the definition.
It's actually not that important
for the level I want to tell you about this project.
The high level idea is you take your algorithm should be
randomized and by adding noise or being randomized,
reveals very little about the data
because a change in one person's data,
doesn't make much difference in the output you
see in terms of probabilities of different up.
That's a high level if you know that,
then you know enough to follow.
So, let me just skip this thing. It's fine.
Okay.
So, what do we want to address
in this project I'm going to tell you about briefly.
So, differential privacy, it's pretty well
understood and practical I would argue for
answering statistical queries about a data set.
So, what is the meaning of this data set?
Or something like that or different queries
about some complex data
but even being used in the census which is pretty cool.
And the most basic approach is that, okay.
So there's this parameter Epsilon differential privacy,
if you add noise that has
standard deviation like one over Epsilon,
then you're preserving Epsilon
differential privacy intuitively.
So, Epsilon very small,
is very private but lots of noise.
So, we understand all that very well.
Machine learning queries and data is very interesting.
So, there's more recent work.
It's quite active but I would argue it's
not well understood at least not in
the way that statistical queries are.
Because the kind of theory bounds we
get might be quite loose because we're
using these complicated algorithms, right?
So this algorithm is,
take the true number, add some noise, okay?
And then we can analyze how accurate that is.
If you're talking about something like
a stochastic gradient descent with
extra noise added to preserve privacy,
understanding how private and accurate that really is,
it's going to be very difficult
and often these papers are written by theorists.
So, who are great and I love theorists, okay?
But their theorems tend to look like,
I should say, our theorems tend to look like this.
They tend to look like the following.
Let's say like you prove
some general theorem that says you run
our algorithm with privacy level Epsilon and I'll
guarantee you an accuracy Alpha with high probability.
And often, you might
prove something with a consonant that looks like this,
more often you prove a
constant that looks like this, right?
So why is this an issue?
I think it's an important idea.
So people do this in algorithms.
But you put the algorithm in
practice and it runs really well,
and it doesn't matter you know the fact that you couldn't
prove a low constant is not a big deal.
But here, I want to argue it is an issue.
So, let's say you're an engineer at
some data corporation and
you want to run a differentially private,
you want to run this algorithm.
Well, you have to choose the privacy level you're
guaranteeing the people, okay?
And the problem is that theorem
isn't really going to help you that much in
figuring out how you actually choose it.
If you want an Alpha.01,
should you set Epsilon to be 100,000?
That will guarantee you an Alpha.01
but there's no privacy at all basically.
We want Epsilon to be around one or less.
On the other hand, you could probably
could get away with setting up Epsilon to one
because probably there's all these
loose constants in there but
you don't know and it can be hard.
And what if accuracy is critical to
your system so you can't afford to take chances?
Okay. So this project and this is a paper it
nips last year with some wonderful co-authors and Steven.
I'm sorry, and that's the most wonderful of them all.
So the question is,
if we have an accuracy requirement,
can we run a learning algorithm that tries to find
the most private possible output that we can
have without having to
tell it what Epsilon and Alpha to shoot for,
or sorry what Epsilon to shoot for,
it will try to satisfy
this accuracy requirement but be really private, okay?
And so, the setting
is what we call empirical risk minimization.
So, you just have this data set and you
have a last function and you
just try to find an hypothesis
that minimizes loss on the data set, right?
Okay.
So, I'm just going to give you
an example of the idea that we are
deploying and they'll be about it for this project.
So that will be about all of them.
Okay. So, one kind
of private algorithm is called output perturbation.
You compute the true output.
So, for example, this is a hypothesis, let's say,
like a classifier or something that you found by doing
empirical risk minimization on your data set, okay?
And now, you add a bunch of noise to all these numbers.
And now, it's private. Okay. And the difference
here is that we're going to use random
walks to add this noise.
So if you run a random walk for a certain length of time,
you've added a certain variance of
noise to each number and that's fine.
And you get some some output.
Okay, but now,
the key idea is to check how accurate this is.
And if it's not accurate
enough you wanted a certain accuracy level,
then we're going to have to improve
the accuracy somehow, okay?
So, there's this idea,
this step called interactive
above threshold that checks the accuracy.
Sorry, I'm not going to talk about
but you have to worry about
leaking privacy when you do this.
Okay. But let's suppose we check the accuracy.
If it's not accurate enough, the idea is to rewind
the random walk. That's the key idea.
We got it from a similar idea in
a statistics paper on
privacy that kind of does the reverse of this.
So now, let's just rewind to here.
That's on average, less noisy.
Check the accuracy. Is it good enough?
If it's not accurate enough,
so if this hypothesis is not predicting well
enough on the data set, rewind some more.
Eventually, as you approach the left you're going
to approach the true best possible hypothesis.
So you will approach something
that's accurate enough, right?
And that accuracy requirement will tell you when
you can stop, okay?
And now,
it's kind of late in the talk but maybe I can try to
convey this intuition for why this is good.
So the key idea of why this is
good from a privacy standpoint
is what you'd worry about is this process,
this interactive process of
compute some output check its accuracy and so on,
is leaking lots and lots of privacy, right?
So, you'd worry that
all these outputs are
leaking information about the dataset.
And you can actually
analyze it as though in the worst case we
published all these outputs that
would leak possibly lots of information.
Okay.
This turns out not to be true because of the random walk.
So, the idea is,
this noisy estimate here.
So, if you see this noisy estimate,
you're inferring something about the data.
But once I reveal this less noisy estimate,
that noisier one does not give
you any more information about the data.
>> Why? Because of these random walks.
So, if I tell you this number, and this number,
and you're trying to guess
or infer this thing, which is private.
This number will not help you at all.
It's independent condition on this one.
That's the idea of the random walk.
>> [inaudible]
>> Exactly. Yeah. So, that's why,
we saved the result of this
random walks secretly and rewind it.
Instead of just try again with less noise.
Yeah.
>> You can't go to the other way,
supposed you have plenty of accuracy,
but you want more privacy?
>> Yeah. So, that's tough to reason about because,
you've leaked some sort of
information just from the fact that
you decided to increase the privacy,
like increase the noise.
And information leak depended on
this very accurate thing you
computed which was very non-private.
I don't want to say it's impossible,
but to analyze is really difficult. So, we don't know-
>> You have to start earliest in
your algorithm, or you have to start with-
>> The very private thing in them.
>> Made more noise than do you think you need.
>> Yeah, yeah.
>> I was thinking like this whole thing is
like done inside a private box.
>> Yes, that's true. That's true.
>> Or you're kind of assuming that average hack
on the right hand side,
leaks information to this public.
>> Yes.
>> You know like [inaudible] board that is true?
>> Yes. Yes. So, it's not necessarily true.
It's more about how we know how to analyze these things.
Often in privacy, we have
a very hard time understanding
how much information is leaked.
And so we'll say,
well, even if we publish this entire thing,
we can still show,
and we don't know how to prove
a better privacy bound for
the thing that doesn't publish it.
>> It could just-
>> It could-
>> Publish the end result right.
>> Yeah. So, in our analysis,
you could have just published
this hypothesis and the fact that it wasn't good enough.
But you don't say what its accuracy
was, you just say not good enough.
Publish this one, you say not good enough.
Publish this one and say that's our final.
>> Because given the final one,
all the previous one that are useless.
>> Yeah. And so it's kind of getting into,
is it just analysis or is it actually what you're doing.
Yeah. It could be that we're just not good at analyzing.
So, that's the key idea
for this project basically that I wanted to convey.
But you can use this trick in other places as
well or for lots of algorithms. Yes?
>> So, my question were also connected
to Alex's previous question.
I don't worry about, say I
only publish the final result and I don't
worry about the evaluation about
the earlier, noisier results.
Do I gain anything for keeping that random work
versus get them fresh renimis to re-round thing?
>> So, if you care about privacy loss,
this is going to be better than doing
a fresh noise each time.
Well, again,
we're not sure how to analyze like where you're
trying to keep things secret, but yeah.
>> Isn't there a little bit of poorly given permission
just from knowing the fact
that it wasn't close enough to the-
>> Yes. Yeah. So, the fact that
this thing's accuracy was not
below the threshold tells you something.
And that is analyzed
in this part of the puzzle that I didn't talk about.
So, in particular like
you don't want them make these like
billions and billions of checks because that will add up,
but you get some sort of logarithmic dependence
on the number of checks.
So, it's not too worrying.
Okay, so let me
just recap this and finish up, so we don't go over.
So, the general idea of this project is take
some known algorithms for
differentially-private machine learning and
just apply these two tricks,
so the accuracy checks that I didn't talk and
this noise reduction that I did.
And then, these algorithms,
you give them some accuracy goal
and they try to give you an answer that's that accurate,
but as private as it could be. So, that's the idea.
Okay. These are important
like we're lower than the others and that's good.
Theorists whatever, so, some quick future directions.
Yes, strategic behavior and learning,
and not even just
learning but you know there's
paradigm I also think of like,
prediction markets and this paradigm rightly.
There's information out there, we're trying to
gather it and make predictions.
So, this is in the area,
I think this general questions super
interesting and there's tons of directions,
from both practice to theory.
Privacy, I feel that
privacy is moving towards practicality and applications.
There are interesting theory questions,
but they start moving
farther and farther away from practice,
but there's a lot of very practical questions
that we need help with.
So, that we can deploy privacy to help people.
And it's like the time to do that.
Fairness, which I didn't
tell you about any fairness projects,
but I think it's the opposite actually.
We're really struggling in theory land
to get a hand along fairness.
And initially people expected some sort of one size fits
all definition which is what happened in privacy, almost.
And now we're seeing that's not really possible.
And there's tons of interesting directions there too.
Okay. So, I like some very high level closing side.
I think it's really important
when both thinking theoretically,
as well as in practice.
To be thinking about the value and structure of
information in these problems that's really important.
So, the information agents give us is really valuable.
And because of that, it's susceptible
to manipulation, and so on.
I think it's important that these systems
respects these issues like privacy,
fairness, and also strategic behavior.
I like to think it's interesting
to take that perspective that agents have
the right to be strategic and act on their own behalf.
And so we should design things that work well,
taking that into account rather than
slapping them on the wrist for being strategic.
Okay. And this issue of data, ownership, and control,
I think is really important and I hope that we can do
some research in our communities to understand it better.
So, that's it. Thanks.
Đăng ký:
Đăng Nhận xét (Atom)
Không có nhận xét nào:
Đăng nhận xét