My dad (an engineer not a mathematician) would use Newton-Raphson[1] to solve basically any problem that wasn’t very obviously linear. When I was a kid, some of my first programming memories were my dad getting me and my brother to implement Newton-Raphson in basic on an HP85a, getting me to implement Newton-Raphson in rpn on an HP calculator, debugging my dad’s (genuinely revolting) basic program[2] which wouldn’t run but (who would have guessed?) used Newton-Raphson to compute something or other.
He basically learned the one numerical root-finder and how to evaluate basic second derivatives and he was set for life on all the problems a career in chemical and process engineering could throw at someone.
Relatedly, I've found Newton-Raphson is a great example of an algorithm where Knuth's "I have only proven it correct, not tried it" rears its head very prominently. The obvious implementations can work flawlessly on toy examples and then fail miserably on real-world examples.
There's a reason why numeric analysis is still actively studied by research mathematicians. If we could just throw something as simple as newton's method at any nonlinear problem, we'd only need people to learn this once in school and everyone could solve everything.
I think this is true for engineers as well! I enjoy getting to know the "theme" of my favorite coworkers over the years. There was:
* The fellow who always looked for the simplest hack possible. Give him the most annoying problem, he'd pause, go Wait a minute! and redefine it to have a very easy solution. He typed very slowly, but it didn't really matter.
* The one who truly loved code itself. He would climb mountains to find the most elegant, idiomatic way to express any program. Used the very best practices for testing, libraries, all that. He typed very fast.
* The former physicist who spent all his time reading obscure mailing lists on his favorite topics. His level of understanding of problems in his domains of interest was incredible.
I could go on and on! It's such a fun taxonomy to collect. All of these friends were marvelous at solving their particular flavor of problem.
As for myself, I like to think that my "trick" is to spend a long time poking at the structure of a problem. Eventually the solution I was looking for doesn't matter anymore, but the tools I developed along the way are pretty useful to everyone!
* The (brilliant) infrastructure engineer who described his modus operandi as 'I read stuff on Reddit and then try it out.' This engineer is now worth, as a conservative estimate, in the neighborhood of $50 million. So maybe more of us should be doing that.
* Another infrastructure engineer, also very effective, who made a habit of booking an external training session (sometimes a series, weekly) for how to set up and integrate every piece of technology we used.
* An engineer (this one is quite famous, and we have only interacted professionally a few times) who simply wrote the best comments in the world. Every method was adorned with a miniature essay exploring the problem to be solved, the tradeoffs, the performance, the bits left undone. I think about those comments quite often.
As an addendum, though, I will say that the best engineers overall all shared a trait - they kept trying things until they got something working. That alone will take you pretty far.
Yes. I like to think that all of the people above could have solved most of the same problems, albeit in wonderfully different styles, but what really guaranteed success was a commitment to just keep at it.
Edit to add: Still, the styles matter a lot! For one thing, they greatly influence which problems each person is interested in. Also, the style you solve problems with colors what your final output looks like, which is perhaps more obvious in engineering than in mathematics.
For me, it's tracing code/pipelines to figure out how a result was produced, typically in the context of that result being wrong somehow. Go To Definition is the most useful function in any editor.
I'm always surprised by how frequently colleagues don't think to do this and are left helpless.
This reminds me of my further theory that everyone needs one 'heavy' and one 'light' technique. The 'light' technique is something that often works well as a heuristic and can be an effective unit of iteration. The 'heavy' technique is something that you can fall back on in difficult cases, something that can reliably solve hard problems, even if it's slow.
Sometimes the heavy technique is: just ask someone else. ;)
You jest, but that's how my sister gets through life, and it's always fascinated me.
She's incredibly intelligent, but more importantly she's a phenomenal social networker. She always has someone to call to ask about any question or solve any problem that comes up in life, and she's great at connecting these people with each other when they have problems of their own - so they all want to help her with whatever she needs, just to gain access to a pool of people they themselves can talk to.
What do you do with a skillset like that? I honestly don't know - something in leadership, probably, something where finding the right people and setting them to work is the most important skill of the job.
That wasn't in jest. I worked in a place where this was a norm. Nothing was properly documented, instead everyone would just ask and answer questions on chats; somehow, this actually kept velocity high.
Found it really hard to adjust to that. I'm the kind of person that prefers to research things on my own, find hard references and understand context. But there, this was the wrong approach.
It's amazing that a lot of new developers don't know how to use them at all! I'm not even talking about using the command line gdb, but just the basic "Set Breakpoint" feature and attaching to processes.
Feynman wrote in his autobiography that much of his success came from having different mathematical tricks than most of his peers. So when they were stuck, he could sometimes make progress.
Interestingly, one of his famous tricks of computing complex integrals by parameterizing[1] and then differentiating under the integral is known as “Feynman’s trick” in his honour spite of it having been invented by Euler over 250 years before.
He had many tricks in limited area. All his his tricks were limited to classical calculus and nineteenth-century mathematics. He didn't do anything fancy.
His style was always the same, he just mastered it really well.
My lecturer was once unable to solve a problem at the blackboard.
After trying for some time, he said he needed a short break to go back to his office to look at his notes. He brought the notes, and there he found a hint he had written for himself: "Use a trick"
I remember annoying one of our professors at the university. Whenever he was discussing a problem and found me falling asleep, he called my name. I woke up, said Chinese remainder theorem and with like 90% success rate it solved the problem handily. Yes, it was an Algebra class. We were still surprised how well it worked.
One of the answers links https://www.tricki.org/, which describes itself as a 'Wiki-style site with a large store of useful mathematical problem-solving techniques.' no longer maintained, but looks neat.
There is a joke in applied mathematics that we’re like Taco Bell. We all use the same six ingredients, mixing them in different ways.
For myself, I’ve found several techniques I use over and over again. Some of this is a “when you’re a hammer, everything looks like a nail.” But fundamentally there are only a handful of ideas. One professor of mine once said the only groundbreaking result in the past few decades was compressive sensing.
For most of my computer science PhD the “trick” was just to get the inductive definition to work, and then how to tweak it for the next paper. Or, get enough structuret we can do an “abstract nonsense” proof[0].
My dad (an engineer not a mathematician) would use Newton-Raphson[1] to solve basically any problem that wasn’t very obviously linear. When I was a kid, some of my first programming memories were my dad getting me and my brother to implement Newton-Raphson in basic on an HP85a, getting me to implement Newton-Raphson in rpn on an HP calculator, debugging my dad’s (genuinely revolting) basic program[2] which wouldn’t run but (who would have guessed?) used Newton-Raphson to compute something or other.
He basically learned the one numerical root-finder and how to evaluate basic second derivatives and he was set for life on all the problems a career in chemical and process engineering could throw at someone.
[1] https://sheffield.ac.uk/media/31988/download?attachment
[2] He learned to program in FORTRAN and lived by the maxim that a determined FORTRAN programmer can write FORTRAN in any language.
Relatedly, I've found Newton-Raphson is a great example of an algorithm where Knuth's "I have only proven it correct, not tried it" rears its head very prominently. The obvious implementations can work flawlessly on toy examples and then fail miserably on real-world examples.
There's a reason why numeric analysis is still actively studied by research mathematicians. If we could just throw something as simple as newton's method at any nonlinear problem, we'd only need people to learn this once in school and everyone could solve everything.
Nobody has enough memory or patience for third order derivatives so Newton's method (aka Newton Raphson) it is.
I think this is true for engineers as well! I enjoy getting to know the "theme" of my favorite coworkers over the years. There was:
* The fellow who always looked for the simplest hack possible. Give him the most annoying problem, he'd pause, go Wait a minute! and redefine it to have a very easy solution. He typed very slowly, but it didn't really matter.
* The one who truly loved code itself. He would climb mountains to find the most elegant, idiomatic way to express any program. Used the very best practices for testing, libraries, all that. He typed very fast.
* The former physicist who spent all his time reading obscure mailing lists on his favorite topics. His level of understanding of problems in his domains of interest was incredible.
I could go on and on! It's such a fun taxonomy to collect. All of these friends were marvelous at solving their particular flavor of problem.
As for myself, I like to think that my "trick" is to spend a long time poking at the structure of a problem. Eventually the solution I was looking for doesn't matter anymore, but the tools I developed along the way are pretty useful to everyone!
Here are a few more.
* The (brilliant) infrastructure engineer who described his modus operandi as 'I read stuff on Reddit and then try it out.' This engineer is now worth, as a conservative estimate, in the neighborhood of $50 million. So maybe more of us should be doing that.
* Another infrastructure engineer, also very effective, who made a habit of booking an external training session (sometimes a series, weekly) for how to set up and integrate every piece of technology we used.
* An engineer (this one is quite famous, and we have only interacted professionally a few times) who simply wrote the best comments in the world. Every method was adorned with a miniature essay exploring the problem to be solved, the tradeoffs, the performance, the bits left undone. I think about those comments quite often.
As an addendum, though, I will say that the best engineers overall all shared a trait - they kept trying things until they got something working. That alone will take you pretty far.
“The most successful people have failed more times than you have tried”
Yes. I like to think that all of the people above could have solved most of the same problems, albeit in wonderfully different styles, but what really guaranteed success was a commitment to just keep at it.
Edit to add: Still, the styles matter a lot! For one thing, they greatly influence which problems each person is interested in. Also, the style you solve problems with colors what your final output looks like, which is perhaps more obvious in engineering than in mathematics.
For me, it's tracing code/pipelines to figure out how a result was produced, typically in the context of that result being wrong somehow. Go To Definition is the most useful function in any editor.
I'm always surprised by how frequently colleagues don't think to do this and are left helpless.
This reminds me of my further theory that everyone needs one 'heavy' and one 'light' technique. The 'light' technique is something that often works well as a heuristic and can be an effective unit of iteration. The 'heavy' technique is something that you can fall back on in difficult cases, something that can reliably solve hard problems, even if it's slow.
Sometimes the heavy technique is: just ask someone else. ;)
> Sometimes the heavy technique is: just ask someone else. ;)
For a lot of people I know, this is the light technique!
You jest, but that's how my sister gets through life, and it's always fascinated me.
She's incredibly intelligent, but more importantly she's a phenomenal social networker. She always has someone to call to ask about any question or solve any problem that comes up in life, and she's great at connecting these people with each other when they have problems of their own - so they all want to help her with whatever she needs, just to gain access to a pool of people they themselves can talk to.
What do you do with a skillset like that? I honestly don't know - something in leadership, probably, something where finding the right people and setting them to work is the most important skill of the job.
That wasn't in jest. I worked in a place where this was a norm. Nothing was properly documented, instead everyone would just ask and answer questions on chats; somehow, this actually kept velocity high.
Found it really hard to adjust to that. I'm the kind of person that prefers to research things on my own, find hard references and understand context. But there, this was the wrong approach.
For me the heavy technique is integer linear programming.
I’m not a software developer anymore.
Another example: debuggers.
It's amazing that a lot of new developers don't know how to use them at all! I'm not even talking about using the command line gdb, but just the basic "Set Breakpoint" feature and attaching to processes.
[dead]
For programmers I would say graphs or at least thinking in graphs is a common one.
Some would say SAT is also a handy trick but I’ve personally never used it.
Feynman wrote in his autobiography that much of his success came from having different mathematical tricks than most of his peers. So when they were stuck, he could sometimes make progress.
Interestingly, one of his famous tricks of computing complex integrals by parameterizing[1] and then differentiating under the integral is known as “Feynman’s trick” in his honour spite of it having been invented by Euler over 250 years before.
[1] https://zackyzz.github.io/feynman.html
A popular naming convention in Mathematics is to name things after the second person to discover them. The first person is Leonhard Euler.
Feynman used to read his own books. When asked he said, "it's all in here". He used to revise and refresh his own understanding.
He had many tricks in limited area. All his his tricks were limited to classical calculus and nineteenth-century mathematics. He didn't do anything fancy.
His style was always the same, he just mastered it really well.
My lecturer was once unable to solve a problem at the blackboard. After trying for some time, he said he needed a short break to go back to his office to look at his notes. He brought the notes, and there he found a hint he had written for himself: "Use a trick"
I remember annoying one of our professors at the university. Whenever he was discussing a problem and found me falling asleep, he called my name. I woke up, said Chinese remainder theorem and with like 90% success rate it solved the problem handily. Yes, it was an Algebra class. We were still surprised how well it worked.
One of the answers links https://www.tricki.org/, which describes itself as a 'Wiki-style site with a large store of useful mathematical problem-solving techniques.' no longer maintained, but looks neat.
I like the trick of adding zero to an equation.
There is a joke in applied mathematics that we’re like Taco Bell. We all use the same six ingredients, mixing them in different ways.
For myself, I’ve found several techniques I use over and over again. Some of this is a “when you’re a hammer, everything looks like a nail.” But fundamentally there are only a handful of ideas. One professor of mine once said the only groundbreaking result in the past few decades was compressive sensing.
For most of my computer science PhD the “trick” was just to get the inductive definition to work, and then how to tweak it for the next paper. Or, get enough structuret we can do an “abstract nonsense” proof[0].
[0]:https://ncatlab.org/nlab/show/category+theory#AbstractNonsen...
From a comment:
> These methods can be combined. First generalize the problem, making it more complicated. Then simplify along a different axis. – Stig Hemmer
Very relevant to software design too.
A mathematician tricks.always. There is no point to untrick the trickster. Misuse of your time
[dead]