Jekyll2019-09-24T11:20:09-04:00https://cs.stanford.edu/people/baxelrod/Brian AxelrodResearch PortfolioBrian Axelrodbaxelrod@cs.stanford.eduNear-optimal Approximate Discrete and Continuous Submodular Function Minimization2019-09-24T00:00:00-04:002019-09-24T00:00:00-04:00https://cs.stanford.edu/people/baxelrod/submodular-approximation<h2 id="abstract">Abstract</h2>
<p>In this paper we provide improved running times and oracle complexities for approximately minimizing a submodular function. Our main result is a randomized algorithm, which given any submodular function defined on \(n\)-elements with range \([−1,1]\), computes an \(\epsilon\)-additive approximate minimizer in \(\widetilde O(n/\epsilon^2)\) oracle evaluations with high probability. This improves over the \(\widetilde O(n^{5/3}/\epsilon^2)\) oracle evaluation algorithm of Chakrabarty et al.(STOC 2017) and the \(\widetilde O(n^{3/2}/\epsilon^2)\) oracle evaluation algorithm of Hamoudiet al. Further, we leverage a generalization of this result to obtain efficient algorithms for minimizing abroad class of nonconvex functions. For any function with domain \([0,1]^n \) that satisfies \(\frac{\partial x_i}{\partial x_j} \le 0\) for all \(i \neq j\) and is L-Lipschitz with respect to the \(\ell_\infty\)-norm we give an algorithm that computes an \(\epsilon\)-additive approximate minimizer with \(\widetilde O(n·poly(L/\epsilon))\) function evaluation with high probability.</p>
<!-- ## Main Idea -->Brian Axelrodbaxelrod@cs.stanford.eduApproximate Submodular Function MinimizationSample Amplification: Increasing Dataset Size even when Learning is Impossible2019-06-11T00:00:00-04:002019-06-11T00:00:00-04:00https://cs.stanford.edu/people/baxelrod/sample-amplification<h2 id="abstract">Abstract</h2>
<p>Given data drawn from an unknown distribution, \(D\), to what extent is it possible to “amplify” this dataset and faithfully output an even larger set of samples that appear to have been drawn from \(D\)?<br />
We formalize this question as follows: an \((n,m)\) \emph{amplification procedure} takes as input \(n\) independent draws from an unknown distribution \(D\), and outputs a set of \(m > n\) “samples”. An amplification procedure is \emph{valid} if no algorithm can distinguish the set of \(m\) samples produced by the amplifier from a set of \(m\) independent draws from \(D\), with probability greater than \(2/3\). Perhaps surprisingly, in many settings, a valid amplification procedure exists, even in the regime where the size of the input dataset, \(n\), is significantly less than what would be necessary to \emph{learn} distribution $D$ to non-trivial accuracy. Specifically we consider two fundamental settings: the case where \(D\) is an arbitrary discrete distribution supported on \(\le k\) elements, and the case where \(D\) is a \(d\)-dimensional Gaussian with unknown mean, and fixed covariance matrix. In the first case, we show that an \(\left(n, n + \Theta(\frac{n}{\sqrt{k}})\right) \) amplifier exists. In particular, given \(n=O(\sqrt{k})\) samples from \(D\), one can output a set of \(m=n+1\) datapoints, whose total variation distance from the distribution of \(m\) i.i.d. draws from \(D\) is a small constant, despite the fact that one would need quadratically more data, \(n=\Theta(k)\), to \emph{learn} $D$ up to small constant total variation distance. In the Gaussian case, we show that an \(\left(n,n+\Theta(\frac{n}{\sqrt{d}} )\right)\) amplifier exists, even though learning the distribution to small constant total variation distance requires \(\Theta(d))\) samples. In both the discrete and Gaussian settings, we show that these results are tight, to constant factors. Beyond these results, we formalize a number of curious directions for future research along this vein.</p>
<!-- ## Main Idea -->Brian Axelrodbaxelrod@cs.stanford.eduIs generating new samples as hard as learning?A Polynomial Time Algorithm for Log-Concave Maximum Likelihood via Locally Exponential Families [merged]2018-11-07T00:00:00-05:002018-11-07T00:00:00-05:00https://cs.stanford.edu/people/baxelrod/polylogconcavemlemerged<h2 id="abstract">Abstract</h2>
<p>We consider the problem of computing the maximum likelihood multivariate log-concave distribution for a set of points. Specifically, we present an algorithm which, given \(n\) points in \(\mathbb R^d\) and an accuracy parameter \(\epsilon >0 \), runs in time \(\mathrm{poly}(n,d,1/\epsilon)\), and returns a log-concave distribution which, with high probability, has the property that the likelihood of the n points under the returned distribution is at most an additive ϵ less than the maximum likelihood that could be achieved via any log-concave distribution. This is the first computationally efficient (polynomial time) algorithm for this fundamental and practically important task. Our algorithm rests on a novel connection with exponential families: the maximum likelihood log-concave distribution belongs to a class of structured distributions which, while not an exponential family, “locally” possesses key properties of exponential families. This connection then allows the problem of computing the log-concave maximum likelihood distribution to be formulated as a convex optimization problem, and solved via an approximate first-order method. Efficiently approximating the (sub) gradients of the objective function of this optimization problem is quite delicate, and is the main technical challenge in this work.</p>
<h2 id="main-idea">Main Idea</h2>
<p>The solution to a log-concave MLE problem is always a distribution whose log-density is a polytope with corners at the observed points \(X_1,…,X_n\). These “tent-distributions” are distributions on the boundary of log-concavity that don’t waste density on points outside \(X_1,…,X_n\).</p>
<p>In this paper we show that tent-distributions are “locally-exponential”. That is, sufficiently small balls in parameter-space always form an exponential family. In fact, tent distributions can be partitioned into a finite collection of exponential families where each exponential family corresponds to a regular subdivision of \(X_1,…,X_n\). We also show a reduction from the log-concave MLE problem to the tent-distribution MLE problem.</p>
<p>We also show that being locally-exponential and satisfying a regularity condition is sufficient to guarantee that a standard gradient descent based algorithm can be used to solve the tent-distribution maximum likelihood problem.</p>
<p>This paper is a merged version of <a href="https://arxiv.org/abs/1812.05524">https://arxiv.org/abs/1812.05524</a> and <a href="https://arxiv.org/abs/1811.03204">https://arxiv.org/abs/1811.03204</a></p>Brian Axelrodbaxelrod@cs.stanford.eduAn algorithm to compute the log-concave MLE in time polynomial in number of samples and dimension that uses the local exponentiality of tent-distributions.An Efficient Algorithm For High-Dimensional Log-Concave Maximum Likelihood [original]2018-11-06T00:00:00-05:002018-11-06T00:00:00-05:00https://cs.stanford.edu/people/baxelrod/polylogconcavemle<h2 id="abstract">Abstract</h2>
<p>The log-concave maximum likelihood estimator (MLE) problem answers: for a set of points \(X_1,…X_n \in \mathbb R^d\), which log-concave density maximizes their likelihood? We present a characterization of the log-concave MLE that leads to an algorithm with runtime \(poly(n,d, \frac 1 \epsilon,r)\) to compute a log-concave distribution whose log-likelihood is at most \(\epsilon\) less than that of the MLE, and \(r\) is parameter of the problem that is bounded by the \(\ell_2\) norm of the vector of log-likelihoods the MLE evaluated at \(X_1,…,X_n\).</p>
<h2 id="main-idea">Main Idea</h2>
<p>The solution to a log-concave MLE problem is always a distribution whose log-density is a polytope with corners at the observed points \(X_1,…,X_n\). These “tent-distributions” are distributions on the boundary of log-concavity that don’t waste density on points outside \(X_1,…,X_n\).</p>
<p>In this paper we show that tent-distributions are “locally-exponential”. That is, sufficiently small balls in parameter-space always form an exponential family. In fact, tent distributions can be partitioned into a finite collection of exponential families where each exponential family corresponds to a regular subdivision of \(X_1,…,X_n\). We also show a reduction from the log-concave MLE problem to the tent-distribution MLE problem.</p>
<p>We also show that being locally-exponential and satisfying a regularity condition is sufficient to guarantee that a standard gradient descent based algorithm can be used to solve the tent-distribution maximum likelihood problem.</p>Brian Axelrodbaxelrod@cs.stanford.eduAn algorithm to compute the log-concave MLE in time polynomial in number of samples and dimension that uses the local exponentiality of tent-distributions.Hardness of 2D Motion Planning Under Obstacle Uncertainty2018-11-05T00:00:00-05:002018-11-05T00:00:00-05:00https://cs.stanford.edu/people/baxelrod/shadow-hardness-ijrr<h2 id="abstract">Abstract</h2>
<p>We consider the problem of motion planning in the presence of uncertain obstacles, modeled as polytopes with Gaussian-distributed faces (PGDF). A number of practical algorithms exist for motion planning in the presence of known obstacles by constructing a graph in configuration space, then efficiently searching the graph to find a collision-free path. We show that such a class of algorithms is unlikely to be practical in the domain with uncertain obstacles. In particular, we show that safe 2D motion planning among PGDF obstacles is $NP-$hard with respect to the number of obstacles, and remains $NP-$hard after being restricted to a graph. Our reduction is based on a path encoding of MAXQHORNSAT and uses the risk of collision with an obstacle to encode variable assignments and literal satisfactions. This implies that, unlike in the known case, planning under uncertainty is hard, even when given a graph containing the solution.
We further show by reduction from \(3\)-SAT that both safe 3D motion planning among PGDF obstacles and the related minimum constraint removal problem remain \(NP\)-hard even when restricted to cases where each obstacle overlaps with at most a constant number of other obstacles.</p>
<h2 id="main-idea">Main Idea</h2>
<p>This paper aims to answer “how much harder is motion planning under obstacle uncertainty than motion planning with known obstacles?” Even though the motion planning problem with known obstacles is known to be PSPACE-hard, we have many practically effective algorithms for generating motions plans.</p>
<p>In light of this, how does one show computational hardness results that provide insight to the real hardness of the problem and guide the development of algorithms to solve it? Simply showing that the new problem is PSPACE-hard (or even a stronger result) would not provide any new information.</p>
<p>We attempt to solve this problem by showing the following things:</p>
<ol>
<li>Graph based methods for solving motion planning rely on the motion plan being captured in a graph of small size. If this graph can be identified, motion planning can be reduced to a shortest path problem which is easy to solve in polynomial time. We show that this method does <strong>cannot</strong> work with obstacle uncertainty.</li>
<li>The reason that the above method works in the known obstacle case is that the system is memoryless. It does not matter how you got to a particular state. This does not hold true for the unknown obstacle case. We show how this property leads to the hardness result. In other words, motion planning with uncertainty is harder than motion planning with known obstacles because it matters which trajectory was chosen to get a configuration.</li>
</ol>
<p>We then proceed to show two results.</p>
<ol>
<li>
<p>Motion planning with estimated obstacles remains hard when restricted to a point robot in two dimensions, PGDF obstacles. It remains hard when restricted to searching a graph as well.</p>
</li>
<li>
<p>Motion planning in estimated obstacles in 3 dimensions remains hard even when each obstacle overlaps with at most a constant number of other obstacles. The result relies on converting a 3SAT instance to a graph embedded in \(\mathbb R^3\) paired with a set of random obstacles. Navigation through the graph forces the robot to go through two stages. In the first the robot is forced to choose which obstacles it must go near. This corresponds to choosing an assignment of variables in the 3SAT. The second state can only be successfully navigated if the right path was chosen in the first stage. The above method relies on constructing the graph and random obstacles such that collisions between the first and second stages are correlated in just the right way.</p>
</li>
</ol>
<p>While the construction presented in this paper conforms to reasonable conditions, all hope is not lost. The result suggests that it is really the long term interactions between collision probabilities that make the problem hard. It suggests several paths towards efficient planning with obstacle uncertainty:</p>
<ol>
<li>Finding a locally optimal solution</li>
<li>Assuming the problem has limited long-term interactions between nodes in the graph.</li>
</ol>Brian Axelrodbaxelrod@cs.stanford.eduThis work quantifies the hardness of 2D and 3D motion planning with uncertain obstacles.Hardness of 3D Motion Planning Under Obstacle Uncertainty2018-11-04T00:00:00-04:002018-11-04T00:00:00-04:00https://cs.stanford.edu/people/baxelrod/shadow-hardness-wafr<h2 id="abstract">Abstract</h2>
<p>We consider the problem of motion planning in the presence
of uncertain obstacles, modeled as polytopes with Gaussian-distributed
faces (PGDF). A number of practical algorithms exist for motion planning
in the presence of known obstacles by constructing a graph using randomly
sampled vertices, then efficientlyciently searching the graph to find a collision-
free path.We show that such a class of algorithms is unlikely to be efficientlycient
in the domain with uncertain obstacles. In particular, we show that safe
3D motion planning among PGDF obstacle is NP-hard with respect to
the number of obstacles, and remains NP-hard after being restricted to
a graph. Our reduction is based on a path encoding of 3-SAT and uses
the risk of collision with an obstacle to encode the variable assignment.
This implies that, unlike in the known case, planning under uncertainty
is hard, even when given a graph containing the solution.</p>
<h2 id="main-idea">Main Idea</h2>
<p>This paper aims to answer “how much harder is motion planning under obstacle uncertainty than motion planning with known obstacles?” Even though the motion planning problem with known obstacles is known to be PSPACE-hard, we have many practically effective algorithms for generating motions plans.</p>
<p>In light of this, how does one show computational hardness results that provide insight to the real hardness of the problem and guide the development of algorithms to solve it? Simply showing that the new problem is PSPACE-hard (or even a stronger result) would not provide any new information.</p>
<p>We attempt to solve this problem by showing the following things:</p>
<ol>
<li>Graph based methods for solving motion planning rely on the motion plan being captured in a graph of small size. If this graph can be identified, motion planning can be reduced to a shortest path problem which is easy to solve in polynomial time. We show that this method does <strong>cannot</strong> work with obstacle uncertainty.</li>
<li>The reason that the above method works in the known obstacle case is that the system is memoryless. It does not matter how you got to a particular state. This does not hold true for the unknown obstacle case. We show how this property leads to the hardness result. In other words, motion planning with uncertainty is harder than motion planning with known obstacles because it matters which trajectory was chosen to get a configuration.</li>
</ol>
<p>The result relies on converting a 3SAT instance to a graph embedded in \(\mathbb R^3\) paired with a set of random obstacles. Navigation through the graph forces the robot to go through two stages. In the first the robot is forced to choose which obstacles it must go near. This corresponds to choosing an assignment of variables in the 3SAT. The second state can only be successfully navigated if the right path was chosen in the first stage. The above method relies on constructing the graph and random obstacles such that collisions between the first and second stages are correlated in just the right way.</p>
<p>While the construction presented in this paper conforms to reasonable conditions (is robust to changing the obstacle distribution, requires only 3 dimensions and a point robot) all hope is not lost. The result suggests that it is really the long term interactions between collision probabilities that make the problem hard. It suggests several paths towards efficient planning with obstacle uncertainty:</p>
<ol>
<li>Finding a locally optimal solution</li>
<li>Assuming the problem has limited long-term interactions between nodes in the graph.</li>
</ol>Brian Axelrodbaxelrod@cs.stanford.eduThis work quantifies the hardness of 3D motion planning with uncertain obstacles.Provably Safe Robot Navigation with Obstacle Uncertainty (Journal Version)2018-06-21T00:00:00-04:002018-06-21T00:00:00-04:00https://cs.stanford.edu/people/baxelrod/shadows-ijrr<h2 id="abstract">Abstract</h2>
<p>As drones and autonomous cars become more widespread it is becoming increasingly important that robots can operate safely under realistic conditions. The noisy information fed into real systems means that robots must use estimates of the environment to plan navigation. Efficiently guaranteeing that the resulting motion plans are safe under these circumstances has proved difficult. We examine how to guarantee that a trajectory or policy is safe with only imperfect observations of the environment. We examine the implications of various mathematical formalisms of safety and arrive at a mathematical notion of safety of a long-term execution, even when conditioned on observational information. We present efficient algorithms that can prove that trajectories or policies are safe with much tighter bounds than in previous work. Notably, the complexity of the environment does not affect our methods ability to evaluate if a trajectory or policy is safe. We then use these safety checking methods to design a safe variant of the RRT planning algorithm.</p>
<h2 id="main-idea">Main Idea</h2>
<p>In order for robots to operate safely and efficiently in the presence of noisy sensor data robots must be able to quantify the accuracy of their estimates of the environment. Especially with low-cost, noisy sensors todays robots can run into walls and other obstacles because they are unable to quantify how far they should stay away from the noisy estimates of these obstacles.
<img src="https://what-if.xkcd.com/imgs/a/5/robot_apocalypse_door.png" alt="A robot that didn't reason about obstacle uncertainty" /></p>
<p>This paper formalizes the notion of a “shadow” as a generalization of confidence intervals that can be used to analyze uncertain shapes. The math developed to derive shadows ends up being useful for more than just providing formal guarantees. It suggests an algorithm for bounding the probability of collision for a given trajectory. This algorithm preserves strong theoretical guarantees, runs exponentially faster than its monte-carlo based equivalents, and fixes issues with previous ad-hoc versions of shadows that would cause excessively conservative behavior.</p>
<p>The math behind shadows turned out surprisingly beautiful and insightful and will be explored in greater depth in my thesis (coming soon!).</p>
<!--![](https://what-if.xkcd.com/imgs/a/5/robot_apocalypse_comparison.png)-->Brian Axelrodbaxelrod@cs.stanford.eduThis work examines how to bound random geometry. It shows how to compute confidance intervals for shapes and applies it to planning under uncertainty. It further develops the related mathematical tools in a way that is better motivated and easier to understand.Running gummi (or other X applications) using WSL2017-07-20T00:00:00-04:002017-07-20T00:00:00-04:00https://cs.stanford.edu/people/baxelrod/gummi<p>Just some notes on how to get GUI applications running on Windows Subsystem for Linux.</p>
<h3 id="prerequisites">Prerequisites</h3>
<p>On windows install an X-Server (ex)<a href="https://sourceforge.net/projects/vcxsrv/">VcXsrv</a>. If you have a high resolution screen, right click on the link created on your desktop and go to properties. In the compatibility tab press the Override high DPI scaling behavior. Set “scaling performed by” to “application”.</p>
<p>In bash install gummi and latex.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo apt-get update
sudo apt-get install gummi texlive-full
</code></pre></div></div>
<p>Download and place the following <a href="/people/baxelrod/assets/other/gummi.zip">files</a> in C:\Program Files\gummi.</p>
<p>I didn’t sign the files so you’re going to want to open and save the two batch files so that the Windows security features let you run them.</p>
<p>Try running the StartX script and allow X-Server in the Windows firewall.</p>
<h3 id="how-it-works">How it Works</h3>
<p>The way we’re going to open latex files is as follows:</p>
<ol>
<li>We’re going to tell windows to open latex files with a batch script I wrote</li>
<li>The batch script will check if the X-Server is running. If not it will start the X-Server.</li>
<li>The batch script will then tell bash to start gummi, passing the location of the file</li>
</ol>Brian Axelrodbaxelrod@cs.stanford.eduHow to get linux GUI applications working on windows using WSLProvably Safe Robot Navigation with Obstacle Uncertainty (Conference Version)2017-05-29T00:00:00-04:002017-05-29T00:00:00-04:00https://cs.stanford.edu/people/baxelrod/shadows<h2 id="abstract">Abstract</h2>
<p>As drones and autonomous cars become more widespread it is becoming increasingly important that robots can operate safely under realistic conditions. The noisy information fed into real systems means that robots must use estimates of the environment to plan navigation. Efficiently guaranteeing that the resulting motion plans are safe under these circumstances has proved difficult. We examine how to guarantee that a trajectory or policy is safe with only imperfect observations of the environment. We examine the implications of various mathematical formalisms of safety and arrive at a mathematical notion of safety of a long-term execution, even when conditioned on observational information. We present efficient algorithms that can prove that trajectories or policies are safe with much tighter bounds than in previous work. Notably, the complexity of the environment does not affect our methods ability to evaluate if a trajectory or policy is safe. We then use these safety checking methods to design a safe variant of the RRT planning algorithm.</p>
<h2 id="main-idea">Main Idea</h2>
<p>In order for robots to operate safely and efficiently in the presence of noisy sensor data robots must be able to quantify the accuracy of their estimates of the environment. Especially with low-cost, noisy sensors todays robots can run into walls and other obstacles because they are unable to quantify how far they should stay away from the noisy estimates of these obstacles.
<img src="https://what-if.xkcd.com/imgs/a/5/robot_apocalypse_door.png" alt="A robot that didn't reason about obstacle uncertainty" /></p>
<p>This paper formalizes the notion of a “shadow” as a generalization of confidence intervals that can be used to analyze uncertain shapes. The math developed to derive shadows ends up being useful for more than just providing formal guarantees. It suggests an algorithm for bounding the probability of collision for a given trajectory. This algorithm preserves strong theoretical guarantees, runs exponentially faster than its monte-carlo based equivalents, and fixes issues with previous ad-hoc versions of shadows that would cause excessively conservative behavior.</p>
<p>The math behind shadows turned out surprisingly beautiful and insightful and will be explored in greater depth in my thesis (coming soon!).</p>
<!--![](https://what-if.xkcd.com/imgs/a/5/robot_apocalypse_comparison.png)-->Brian Axelrodbaxelrod@cs.stanford.eduThis work examines how to bound random geometry. It shows how to compute confidance intervals for shapes and applies it to planning under uncertainty.Reducing FPGA Algorithm Area by Avoiding Redundant Computation2015-05-26T00:00:00-04:002015-05-26T00:00:00-04:00https://cs.stanford.edu/people/baxelrod/fpga<h2 id="abstract">Abstract</h2>
<p>We develop a new paradigm for designing fully streaming, area-efficient FPGA implementations of common vision algorithm building blocks. By focusing on avoiding redundant computation we are able to get 1-2 orders of magnitude reduction in design area utilization as compared to previous implementations. We demonstrate that our design works in practice by building five 325 frames per second high resolution Harris corner detection cores onto a single FPGA.</p>
<h2 id="main-idea">Main Idea</h2>
<p>Simply implementing algorithms designed for a CPU on a FPGA doesn’t work very well. Underlying the issue is that on an FPGA one must worry about gate complexity in addition to time complexity. Algorithms that have a lot of branching (conditional statements such as if) might have low time complexity but require a very large circuit to implement on an FPGA.</p>
<p>In this paper we examine a pair of common computer vision operations (non-max suppression and convolution). By analyzing the algorithm as a directed graph with computation at the nodes we can gain insight into the gate complexity of the algorithm. Minimizing the number of nodes in the graph allows us to design fast streaming algorithms for these two operations using many fewer gates. With window size \(n\) we get resource complexities of \(O(n)\) and \(O(\log n)\) for low-rank convolution and non-max suppression respectively. We demonstrate the designs by implementing a <a href="https://en.wikipedia.org/wiki/Visual_odometry">visual odometry</a> pipeline with very high performance.</p>Brian Axelrodbaxelrod@cs.stanford.eduThis work examines common computer vision operations and aims to bridge the gap between the time complexity of state of the art CPU algorithms resource complexity of state of the art FPGA algorithms.