-(Male recorder) I just wanted to say one word of..There was some discussion..You're apologizing a little
maybe for not talking about neural nets
and so on. And so Brandon and I are mostly
responsible for-for choosing the speaker
with students and other faculty.
And-and what we had thought about for 
this session was to invite a set of
people who are probably the best
examples of people who are working
across linguistics and the NLP machine
learning ACL community so we wanted this
this to be not just about perceptrons
and syntactic structures, but more
broadly about ways of building
connections and-and understanding
the differences between the work being
done in linguistics and natural language
processing and so on. So if to the extent
that we can talk more about that sort of
thing during this session, that'd be
great. yeah what are the barriers to
-(Male voice) Joe, that sounded close to a question. You want to rephrase that..
-(Male recorder) Yea actually that would work right? Unless Brandon you had something you wanted to..no.
Yea!
[AUDIENCE LAUGHTER]
What are-What are...
-(Male speaker) Oh why don't you ask...
-(Male recorder) Hear this! What are the barriers to better connections
between linguistics and-and the NLP community?
-(Male speaker) Oh that's a good question.
-(Male recorder) And also what are the things that we can get by building those bridges?
What are the reasons for overcoming those
barriers? I am..sometimes it has already come up
in the talks, but if we could kind of sum
up and maybe go a little further with
that I think that would be useful yeah.
-(Female speaker 1) I..so...
-(Male speaker) Go ahead.
-(Female speaker 1) I get into this on a quarterly basis on Twitter with people
and one of the things that often comes up in the direction of why is linguisitics not better informing NLP
is that people say, "oh well you know I looked at
the linguistics classes and none of them
were approachable to me and so I can't",
and so my answer is well okay I
wrote a book for you, but also go
talk to people. Go build the
collaboration so I think one big venue
is events like this one and I think also
the bill to break it and tasks and
others that actually bring people
together as to catalyze collaborations
because I think we do need linguistic
expertise and possibly the best way to
do it is by actually having a linguist
in the discussion helping to design
tasks, helping to do error analysis.
And then the why of that is thinking about
it the benefits to NLP is better NLP.
-(Male speaker) So let me know...
Let me express the worry about that..maybe there's a solution to the worry.
So the solution might be some kind of matchmaking service between people who understand collaborating
because I think..I don't know I get inquiries frequently from people who would like to recruit me as
an NLP person to work on their political
science project or their understanding
medical records project or something
like that. Nobody has time to get
interested in everybody else's
interesting tasks and I think it's
difficult for people to find people who are willing to collaborate with more than just having lunch bases.
-(Female speaker 2) Yeah I mean...
-(Male speaker) It's clearly the right thing to do I mean I agree I agree with that.
-(Female speaker 2) I think sort of related to that maybe this is kind of obvious, but
I actually think the main barrier is
time and not just in terms of
collaboration with senior people but I think for
students. I think you know it's they're
sort of getting more and more..well I
don't know actually maybe in some ways
maybe with neural networks there's
actually less that you have to know in
order to actually get something to work
then there was five years ago when you
had to know a lot more, but I'm not sure if
that's true. But you know there's a lot of things that you need to know on
a technical side. That is on the
technical you know machine learning and
mathematics side, but there's also you
know to really understand linguistics
there's a lot of technical linguistics
things you need to understand as well.
You know? And I think people like myself
who were lucky enough to sort of start
off as a..in my first year of college
basically studying both you know I've
had a very long time to get to that
point, but I think a lot of people
discover one or both of these fairly
late and maybe you know as a result it
really is hard to get both of them. But I don't know if there is a great solution for that.
-(Male Speaker) Sorry. Go ahead.
-(Male recorder) Yea I was going to say something that connects these two things.
And that's that in terms of..so in order to
have a collaboration there has to be a
reason for both people involved in
collaboration to want to do it right?
But you can't just duplicate one another's
ability and so I guess the one thing I
found in practice myself from
collaborating with people in other
disciplines is you actually do have to
learn something about their discipline
you actually..and you have to be able to
do a little bit of it at least yourself.
So before I could start working with
people like Brandon, I at least had to
get the perceptron update rule to work
because that's very simple but at least
I knew something. And I think it
goes both ways right? Yes those two
things seem to be important to me having
shared interests, having shared goals and
I'm really really encouraged by this
something's come up a couple times in
this workshop the idea of building new
tasks with linguists input seems to me
to be a way in which we can be
simultaneously hitting the goals of
linguistic theory and the goals of
natural
language processing and I'm really
encouraged to see that I mean Sam's work
in another work like that this seems to
be taken on and the work from from
Hopkins, Macodrol, and in your group of
sending up tasks that the linguist and
NLP people find interest in and this just seems like..and is this relatively new?
I mean is this something..this..is this a
new trend or...?
-(Male speaker) Well there's stuff like you
know the Twitter Brad challenge so they
you know basically other tests to try to
break things by showing that they..Sort of 
like Smolensky's examples that Tom was
talking about. So you can do the same thing.The business of trying to be adversarial is very hot in the
machine learning community where you
know every paper..other paper that
nips is about generative adversarial
networks where you have two networks trying to do the task
while the other is generating examples to fool it.
And it's possible that we don't even have to set up tasks if we just try it again.I don't know.
-(Female speaker 1) I get...
-(Male audience) So on this point of interdisciplinary collaboration, I think one thing that's worth keeping
in mind is that I think that the time is right to revisit this sort of computational linguistics
as in involving both computer science people and linguists because NLP has grown so much in recent
years like the ACL was nearly 2,000 people this year so there are a lot of people around looking for stuff to do
and they don't all have to work on the same problems, but also I think we should be hearten by the fact that
NLP researchers are more and more finding ways to do
interdisciplinary collaboration with
other disciplines so you know people working with political science scientists, with economists,
with psychologists and not just the social sciences.
We see people collaborating
outside in other areas of computer
science like software engineering. Some
of the papers Emily mentioned in her
talk were kind of in that vein so..and in
robotics and you know there's this huge
language and vision thing which which
comes up somewhat in Jacob's talk. So I
think NLP is opening up more than it used
to be and it's as a field less insular
and linguistics should be privileged in
you know among the many people we could
we can collaborate with in NLP
linguistics should be privileged but
you're gonna have to fight for people's attention,
but there are more people. You know the pond is bigger.
-(Female speaker 1) Yea I mean to respond to that. If every single NLP
research group said "oh we need a
linguist", there probably wouldn't be enough
the linguist or at least not enough willing
linguist because the ACL has gotten so big.
-(Male audience) I just have a general question on this line. So what about you know linguistics coursework
for computer science grad students? 
I've noticed lots of the researchers or
at least people were getting PhDs at a
similar time as me I've noticed people
doing linguistically informed work. It's often
your linguistics training tends to be
rather informal maybe this is really biased
by the places I've studied or whatever
we're at the point where like informal
training doesn't scale like you know
hundred students come in wanting intro
to NLP like you need linguistics courses
to train people in it.
Is this a good model? Should there be
everyone be taking linguistics courses? I
don't know.
-(Male recorder) Can I just share an anecdote? Jacob and I were skiing recently and he told me that most people
in Dan Klein group go over and take
syntax at Berkeley. But this isn't always
the case for it. I think particular instance and I think it'd be
really interesting to have a con..I don't know-think we have the time for it here but a conversation about what
the barriers are to having that happen more? Why?
-(Male speaker) The barriers are usually the reluctant PhD students  to take more classes than they have to
and the fact that they are already required to take a lot but people here will be interested to
know that at Johns Hopkins when Fred
Jelinek was in charge of CLSP, we
maintained a pact among the faculty for some
years that every..as advisors we would
make all our PhD students take at least two linguistics courses.
-(Male recorder) Nice.
-(Male speaker) Syntax,Maddox, Penology take any two and that was a really good thing
so the..after Fred passed away and you
know the center grew in various ways, I
think that's sort of lapse. But it was a
really good thing for creating a culture
where people had shared terminology and
understood these interests. There's
another way of doing it, which is to
ensure that those of us who teach NLP
courses are including a reasonable amount of linguistics, which I know Tom can speak to whether..
he took my class this..he can speak to whether or not I succeeded in doing that but...
my guess is that some NLP courses are
fairly applied. So you know those of us
who are up here are from a generation
which is you know old enough to get
invited to give talks like this but I'm also you know old enough that there were fewer NLP courses
and a lot less machine learning to know and so we took a linguistics courses. You know I took...
-(Both female speakers) I agree with this!
-(Male speaker) yea well I took..got a degree in computer science but I took a lot more linguistics courses
during that because I knew computer science. I took ten linguistics courses.
-(Male recorder) Yeah.
But-but it's gotten..the problem is the growth of all
the fields.
So we have a question way in the back
-(female audience) Yes. I am (inaudible). So this is just an observation not really solution to the problems.
We are hiring so hiring a computational linguist. So an observation I had is actually among the application files
very few if not zero actually from computer science department. We mostly have computational linguists
from linguistics department, few comes from comp sci programs, but really we are hoping sort of..
We are really very open with trying to find a computational linguist that can help
other linguists and tell us what is going on in other fields and what can we learn and
what can we sort of help with other fields. We are trying to really keep our eyes open to find this person that
doesn't necessarily sort of is a linguists in a traditional sense but it sort of surprise me and my colleagues that
we actually get very few applicants from the computer science department. Just wondering about this culture like
what is sort of the uhm people talk about collaborations but what you see
is this sort of psychological distance. Always about maybe salary related or whatever.
[AUDIENCE LAUGHTER]
this presence of...
-(Male audience) Well I..so I did wanna bring..we've
talked a lot in the last 15 minutes
about what sort of what NLP should do to be more linguistics, but I actually think the other direction is much
is equally important which is what linguistics should be doing so they know more machine
learning because there are a lot of
really fundamental questions that are
asked and answered in machine learning
that of crucial interest to the
enterprise of linguistics that just are
completely underappreciated.
And so why don't linguists taking a
class beyond like data analysis or something like that?
-(Male recorder) Phonologists are.
-(Male audience) Should we be changing that? And so this is I think maybe speaks to this question
that uhm of you know well where would we expect to find sort of computational linguists that maybe they are being trained...
that they are being trained maybe it was just...
-(Female audience) Right. To address your question, it's not like we are fully aware we need expertise
in the area to help with the program that ask questions that we don't have (inaudible)...
So that's what I am asking. What was the barrier such that people (inaudible) jobs like that?
-(Female speaker 2) So I realized that my being on the job market is now you know apparently
out of date, but actually as someone
who..I think of myself in some ways as
more of a computer scientist than a
linguist. Although actually my PhD is in
linguistics and it was in the linguistic department, but when I went on the job
market I actually was not applying to
computer science departments because I
wanted a more sort of interdisciplinary
view of things. Something that I was
surprised by when I actually went to
some departments which were linguistics
was the linguistics departments that I
had sort of grown up in were all ones
that were very interdisciplinary and
very open to computation. And then
despite the fact that other places were
advertising for computational linguists
I found that when I went there, I did not
feel they were actually open to
computation so there might be one or two
people that were and everyone else I
felt like I couldn't have a..it was very
hard for me to have a conversation with
them so I do think that that may be
part of it and I think things have changed
since then. I think many more departments
now are you know have have a stronger
computational or at least empirical
focus than they did at that point, but I
do think that's part of it. I also think
to be perfectly honest you know part of
it is just you know even in computer
science departments it's hard to hire
NLP people I mean there's a huge demand.
And so if there's going to be any sort
of question of you know "oh I'm probably
going to get 10 top offers and I could
go get a job at Google you know why
bother applying to a linguistics
department if I think there's going to
be more of a communication gap".
I think that maybe part of it.
-(Male speaker) Just one quick addition to that the..so machine learning is a really excited..is a really
exciting time at the moment. There's a
lot of new methods and there's a lot of
success in different tasks. And somebody
who is interested in both mathematical
modeling machine learning and in 
language may want to preserve some
flexibility if it doesn't work out. So
you know I'm a student and I'm applying
somewhere for grad school if I go to
computer science department if language
doesn't work out, I can take the same
toolbox in applied division.
Where if I'm in the linguistics department of
computational linguistics doesn't work out
what am I going to switch to? Field
work?
-(Female speaker 1) Maybe. There are some people with that pedigree.
-(Male speaker) No there are yeah, but I think it's easier for people
to..I mean there's sort of within the machine
learning community there's a lot of
people who are sort of mercenary about
what tests they apply their toolbox to
and maybe that's part of the problem
it's saying all these mercenaries at ACL,
but it does make a computer science
department sort of an attractive
place to be. If you don't get the
academic job, you can go and you know
work on any number of interesting things
in industry not just (inaudible).
-(Female speaker 1) Picking up on mercenaries and ACL, I wanted to address Chris's point which I
think got a little bit lost here about
paying attention and I think a little
bit of that just like the NLP
people tell me, "Well your syntax classes
are impenetrable. I'm not going to take
it". I think that a lot of the ML
literature I mean there's you know..yeah
it's full of equations, but that's
actually not the bigger problem. The
bigger problem is all of the over
claiming, so you can't really find the
signal for the noise. Everybody who says
you know in the introduction talks about
what babies do and then goes on to
something that's got absolutely nothing
to do for babies do. How am I gonna find
the papers that are actually interesting?
-(Male recorder) Were you talking about linguisitics or computational and computer science?
[AUDIENCE LAUGHTER]
-(Female speaker 1) You're funny.
Uh we should probably wrap up, but so we're
gonna keep doing this. We had a very
productive business meeting the other day.
Please join the the SCL Society
for computational linguistics mailing
list if you'd like to keep up with..we'll be sending out more information.
I'll put everyone who was on the
easy-chair group. I'll put you all on
that and then you can get yourself off
the mailing list if you need to.
I wanted to thank Coral Hughto,
Andrew Lamont, and Susan Blodgett
students at UMass who did a lot of work
in this and we'll see you next year.
[AUDIENCE CLAPPING]
