- The majority of relatively important scripts are managed on our lab github page VriezeLab Github. Here are some helpful git commands and directions to use github.
- The Minnesota Supercomputing Institute does various weekly in-person tutorials, which you can find here. MSI tutorials are also available on YouTube
- University of Minnesota Liberal Arts Technologies and Innovation Services (LATIS) Research Workshop Series
- If you are brand new to working on the command line check out Codecademy.
- To connect to the wired network you have to register your MAC address.
- Testing regular expression (in javascript or python)
Simple functions & examples
Simple functions we seem to use all the time
Genomic control is a standard correction in genome-wide association studies for population stratification. It was first described (afaik) by Bernie Devlin and Kathryn Roeder in Biometrics. The correction is pretty simple, it basically takes the chi square of the median pvalue (which we expect to be .5 under the assumption of no stratification) and divides it by a chi square of .5. This is easy to do in R:
You can then correct all pvalues by this genomic control:
UNIX basics
Slurm
Give detailed information about jobs recently submitted by a particular user -u svrieze, esp "maxrss" the maximum memory used by a particular job.
OpenMx Example Scripts
OpenMx Example Scripts
Here are example scripts in OpenMx for a variety of models. Most models will utilize twin samples.
OpenMx Optimizer Codes
- A value of −1 means that the optimizer found itself stuck in a location where the objective function could not be calculated, and could not find a way out. This most often happens if the starting values make the calculation impossible. Estimates from runs with output$status 1 equal to -1 can never be trusted, and the job must be re-run with different starting values, or a corrected model specification.
- 0 indicates a successful optimization--no error returned.
- 1 means that an optimal solution was found (like status 0), but that the sequence of iterates did not converge. In practice, this is the same as status 0, but involves slightly different criteria and is distinguished for technical reasons. This code is labeled (Mx status GREEN).
- 2 or 3 means that the box constraints or non-linear constraints, respectively, could not be satisfied.
- 4 means that the iteration limit was reached with no solution found. You can use mxOptions() to set a higher iteration limit, or just run mxRun() using the output—it will restart from the most recent set of estimates, and make another run of the same number of iterations.
- 5 means that the Hessian at the solution is not convex. There is likely a better solution, but the optimizer is stuck in a region of confusing geometry (like a saddle point).
To put another way thanks to Rob Kirkpatrick: The Hessian matrix (the matrix of second partial derivatives of the fitfunction w/r/t the free parameters) is not positive-definite at the solution. To put it another way, the fitfunction is not locally convex at the solution. This is a sign that the solution does not meet second-order conditions for being a local minimum of the fitfunction.
- 6 means that gradient is not close enough to zero but the optimizer could find no way to improve the estimate. A variety of situations can lead to this outcome. Check whether the model is identified (mxCheckIdentification). It is also possible that gradients are just slightly larger than the arbitrary threshold used by OpenMx. The estimates resulting from this run MIGHT not be optimal estimates, and it would be risky to treat them as solutions without further investigation, so this code is labeled (Mx status RED). Sometimes re-running the model from its solution [firstRun <- mxRun(myModel) followed by secondRun <- mxRun(firstRun)] is sufficient to make code Red disappear.
To put another way thanks to Mike Neale: A value of 6 means that optimality conditions could not be reached, and the optimizer could find no way to improve the estimate. It often implies either a mistake in the model specification or starting values in an intractable range. The estimates resulting from this run are often NOT optimal estimates, and should not be treated as solutions, so this code is labeled (Mx status RED). The job should be re-run, either from the final solution, or from new starting values, or both. If multiple different starting values converge on the same solution (same parameter estimates and fit function) then confidence that the minimum has been found increases.
- 7 means that the analytic derivatives are incorrect. This should be reported to developers.
- 9 means that an invalid parameter was passed to the optimizer. This should be reported to developers.
- 10 means that the parameter vector is outside the feasible set. This most often happens when the starting values are infeasible. Given feasible starting values, optimizer should under no circumstances escape the feasible set.
MySQL
Connecting to MySQL server in bash.
Once connected, some simple MySQL commands to browse and query.
Copy remote table to local file.