My apprehension was that the computer vision community has been suffering some serious cognitive dissonance lately because here they spent all these years mapping problems to feature spaces of manageable dimensionality, backed by theory saying that proper assumptions must be made to reduce the search space; and then comes these deep nets, hardly tailored to the problems, and out-performs algorithms with decade old history of fine-tuning.
Despite this, I don't think anyone disputes the potential of a good set of assumptions. Instead I think what deep learning has thought us is that we should reconsider what these assumptions should be. While geometry might well be the first kind of language a toddler learns to think in, this should probably not be confused with the rigorous geometry of Euclid. Quite possibly we have some spatial relationships such as the affine transformations hard-coded in our brain at birth, but this does not mean, for instance, that one is therefore necessarily ever able to to draw a house in correct perspective.
Yours is a common sentiment but it is not very accurate. Deep learning is not separate from or a more evolved form of machine learning. It's a learning algorithm, with its own structural biases, such as from architecture and optimization method. There is a duality between optimization/search and sampling, they're two sides of the same coin. Instead of giving labels, I prefer thinking about shared/unshared properties and assumptions of the algorithm. Deep learning is unique in its ability to use gradients and credit assignment through a hierarchy when possible.
Deep learning, like all learners, must take assumptions and there are trade-offs. You trade time spent thinking up features (which reduce learning time and improve sample efficiency), for time thinking up architectures and the appropriate biases (graph conv, rnn, convnet, etc). Though still limited to a particular domain, you hope to gain an architecture that works with broader applicability than a feature based method. You also add more complex training, longer experimentation waits and gain a level of indirection between the problem and the learning algorithm. This means that the gains from new knowledge and understanding of a domain filter through slower for deep learning.
In deep learning, it's a very experimental science. Theory is usually after the fact and not very filling. People try this or that and then write a show and tell paper, but knowledge still helps. Where as in shallow methods, learning something about how images work can directly inform your algorithm, in deep learning you now have to think what sort of transformation will best capture the invariant I am looking for.
Deep learning (especially when you can get it end to end) has removed the expertise threshold for impressive results (the bigger your budget the better). But if it were really so magical, the way some people go on about it, then it would already be obsolete as a field because, minimal need for human expertise.
Excellent point -- our brains and deep networks seem to learn a kind of "fuzzy geometry" that is in some ways more robust than numerically correct geometry, and this also allows us to spend more cycles on higher-level abstractions.
My problem with our brains and NN is exactly this "fuzzy geometry" you speak of. Using precise geometry, if we have a pair of cameras equivalent in layout to eyes we can reconstruct a world with mm-precise accuracy. That is something that cannot be done with neural networks (artificial or biological), instead you get a sort of semantic tagging of "green sofa here, table here, bed there".
But if afterwards you want to use this model to know if the sofa will fit into the alcove (or the car into the garage), the NN systems will be wildly unreliable.
It seems to me that humans can predict 3D spaces pretty good.
I've done alot of construction work. After some practice you can measure quite precisely what does fit where and even how long something is. Our brains seem to be able to "compute" such things despite all the difficulties of constructing a coherent image of our surroundings in the first place.
Or think of moving your huge couch through the narrow stairway of your house - we can predict how you need to turn it so it fits. Or think of truck drivers that are able to maneuver their large vehicles within cm range. Even when they can't see some of it (dead angles) and need to rely on their mental model.
Did I misunderstand your comment? Could you elaborate what I am missing?
Anecdotal evidence: I used to work at a construction materials depot when I was younger. I remember how lost I was when I initially started, I couldn't recognize any of the material sizes by looking at them, while all the old timers always just knew what size they were just by glancing at them. Then only after a couple of months, suddenly I could too. Years later I'm still pretty good at guessing lengths, while people around me seem to always be way off. I can hold out my hands and make a foot, three feet, 1 cm, 0,5 cm 10 cm between them pretty reliably, while other people seem to be way off.
You can learn to estimate various geometrical quantities with surprising accuracy, but it always depends on context (scale of the geometry, distance to it and angle from which you view it).
One interesting thing that I recently noticed is that essentially everybody is going to grossly overestimate the angle of slope of road and at least while driving many people perceive curves of less than ~30° as essentially straight road.
I'm talking about the optical illusions such as seeing a small person next to a car makes the car look bigger. Anything based on ML instead of geometry, or even using ML as a strong prior, will be susceptible to this kind of problem.
Wow, this is really fascinating. It's remarkable that the human brain (well, at least my brain) has almost no issue understanding the "scene". I wonder how a NN would do...
The composition of the scenes seems to neatly match the particular way how human vision works, namely, that we look for/at details with only a narrow center part of our vision, and gather the total detailed picture by scanning over it with our eyes.
So, for this type of artwork, whenever we're looking for details, we rest our eyes at a single separate photograph that covers those details fully just as if we'd be looking at a normal real image; but this contrasts with the wider view captured by our peripheral vision, which is obviously artificial.
Hockney was fully aware of this mechanism when he made these works, which motivated him to make them in the first place:
> For his part, critics often got Hockney all wrong as well, misinterpreting the intensity of the ways he would presently be engaging photography—taking literally hundreds of thousands of photos, coming to feel that the Old Masters themselves had been in thrall to a similar optical aesthetic—as a celebration of the photographic over the painterly, and specifically the post-optical painterly, when in fact all along he’d been engaged in a rigorous critique of photography and the optical as “all right,” in his words, “if you don’t mind looking at the world from the point of view of a paralyzed cyclops, for a split second, but that’s not how the world really is.”
But those conventional, mm accuracy detectors tend to work best when they know what to look for. Isn't this, preanalyzing the scene for tailored vision algorithms, where those fuzzy NN classifiers (will) have the biggest utility?
ML vision may get all the fame right now, but useful applications will need both.
I'm not talking about detection, I'm talking about 3D reconstruction and making judgements on whether X will be able to pass by Y without hitting it.
ML has all the same weaknesses that humans do in that they are easily fooled and can't accurately project outside of their trained parameter space. Without an accurate physics model and an integrated comprehension of the geometry involved we are basically planning to put autistic pigeons in charge of driving our cars.
But the nice thing about computers is that you can feed the results of the high level ML classifier into the mm accurate geometry detector. You can only determine whether X will be able to pass by Y if you already know that there is an X and a Y.
Humans and pigeons will identify an X and a Y and will then try to use the same method to classify the scene as either "will pass" or "won't pass". Computers can do the fuzzy thing for identification, but don't have to use the same method for "will pass"/"won't pass". They can switch to geometric measuring and simulation for that.
In order to accurattely ascertain whether thing X will fit into Y hole you have to know accurate geometry of both which is not something that can be reliably determined from results of the fuzzy classification pass. When humans want to know this they either trust their fuzzy judgement and try it or measure the X and Y in question with tape measure.
Except that DL can't provide the same things as a lot of CV research. They are two separate areas of research for different problems.
DL doesn't provide solutions for challenges in robotics or augmented reality that CV is very good at. For example it can't place the camera at a specific position in the world using the image. DL can tell us what's in the image which CV can't. But CV can place the locations of those objects relative to the viewer.
Yes, but if you read the paper you'll see that he's talking about tightening the gap between the Deep Learning approaches and the current features + geometry state of the art, which is an order of magnitude more precise. DL approaches are however quite faster.
Does DL provide mechanisms for feeding back into theory? As in, does a successful deep convolutional neural network provide a means to extract enough from it's structure and behavior to potentially not NEED the CNN for prediction in a future iteration? Gradient projects can be used to gather a total derivative quantity, and compare sensitivities across inputs. We can regularize to prevent overfitting with cross validation, L-curves, etc. But what about hypothesis generation?
For many of us that have dipped our toes in the ML tooling but don't have a great application for it in our work areas, this would is the kind of thing that we would like. A NN that predicts well, AND has a well understood methodology that gives us actual insight and not just a black box.
Maybe what I have in my head is a deterministic gradient-based analog of an evolutionary algorithm? I'm not sure.
Pretty much the same thing has happened in natural language processing. Previously trained linguists would spend lots of time on carefully crafted features. Now you just throw a bidirectional lstm model at the problem and enough training examples and you are close to state of the art.
Despite this, I don't think anyone disputes the potential of a good set of assumptions. Instead I think what deep learning has thought us is that we should reconsider what these assumptions should be. While geometry might well be the first kind of language a toddler learns to think in, this should probably not be confused with the rigorous geometry of Euclid. Quite possibly we have some spatial relationships such as the affine transformations hard-coded in our brain at birth, but this does not mean, for instance, that one is therefore necessarily ever able to to draw a house in correct perspective.