# Grading Programming Exams part 2

(this follows from my last post on this subject)

Now, we’ll formulate this as an LP. Then, I’ll write a Python script to solve it, PySCIPOpt as the solver (and maybe I’ll add support for the Scipy LP solver later).

## Inputs:

• Integer $N$, the number of students
• Integer $n$, the number of criteria
• Target grades $\{t_1, t_2, \cdots, t_N\}$
• Criteria scores $\{c^{j, i}_1, c^{j, i}_2, \cdots, c^{j, i}_n$ for each student $j$
• Bounds on a subset of the weights: $w^{min}_{i} \leq w_{i} \leq w^{max}_{i}$

## Constraints

• $\sum_{i} c^{max}_{i}w_{i} = 100$
• For weights with no specified bounds: $w_{i} \geq 0$
• For weights with specified bounds: $w^{min}_{i} \leq w_{i} \leq w^{max}_{i}$
• For student $j$: $0 \leq \sum_{i} w_{i}c_{j, i} \leq 100$

## Objective

Minimize $\sum_{j}^{N} \Big( t_{j} - \sum_{i}^{n} w_{i}c_{j, i}\Big)^{2}$

## Introduction

I’m a TA for a class on Systems Programming (taught in C). One of the TA responsibilities is to grade the exams. Usually, we will each take a question from the exam, and grade every student for that question. However, this task, especially when you have hundreds of students, is pretty laborious.

It would be nice if I could just give binary grades: either the program works, or it doesn’t. Unfortunately, that wouldn’t be a good reflection of the student’s knowledge, especially on a timed exam: the students who are 90% percent of the way there would get the same grade as the students who barely attempted the problem. Given this, we have to give partial credit based on the quality of the solutions, even if the solution isn’t perfect, or doesn’t compile. In my view, there are two obvious ways of doing this:

1. Simply look at their solutions, and make a judgment about quality (A, B, C, D, or F).
2. Have a list of independent, $\latex m_{i}$-ary criteria $\{c_1, c_2, \cdots, c_n\}$ with a set of weights $\{w_1, w_2, \cdots, w_n\}$ such that the student’s grade is $\sum_{i = 0}^{n} w_i c_i$ and the sum of weights is 100. When I say $m_{i}$-ary, I mean that each criteria $c_i$ can take on a set of $m_{i}$ values. There should be a clear and consistent criteria for deciding the value of $c_i$. Then, there will also be a set of criteria that don’t have a numerical weight, but is a  necessary condition for a certain grade. For example, to get a 100% on a question, the program written must function perfectly.

Of course, the problem with #1 is that it’s difficult to be consistent across exams; this is partially due to the fact that the way I judge the solutions will probably change as I grade the exam, causing a drift in grades.

The problem with #2 is that it isn’t obvious how to come up with the weights, especially before I start grading the exam. As I grade the exam, I will often start with a grading criteria, with weights, and adjust the weights/remove criteria as I notice patterns in the exam. For example, I might start with having one of the criteria be that the program runs and produces the right output (this would be binary), and have that worth 30% of the question. But, if almost no one is able to achieve this, it might make more sense to say that the program working perfectly is a necessary condition of getting 100%, and drop it as a weighted criteria.

In some cases, I may realize that I need to add a criteria, in which case I need to examine the questions I’ve already graded. This is obviously a hassle, and very time consuming.

But, I think I’ve figured out a way of combining #1 and #2 to create a better grading system.

## The System

First, I’ll lay out a grading criteria without pre-specified weights. Each criteria $c_i$ needs to be independent of other criteria, $m_i$-ary, and consistently grade-able.  Then, I will go through each student’s solution $s_j$, and assign it a target grade $t_j$ (100, 95, 90, 85, 80, 75, 70, \cdots,  0).

Then, I’ll throw out the extremes: those that get 100%, and those that get 0s. Then, the task is to assign a weight to each criteria that minimizes mean-squared error. This is basically regression, except we say that each weight must be non-negative, the sum of weights must be 100, and some of the weights can have upper and/or lower bounds.

Also, it would be good to have the weights be integers (though that isn’t strictly necessary).

I can formulate this as an LP. But that’s for another blog post!

# Libscip and PySCIPOpt

(NOTE: I’m reconstructing the order of events in how I fixed this, so there might be some fuzzy details)

I was trying to install PySCIPOpt and the SCIP Optimization Suite, to do some homework with.

I followed the installation instructions on http://scip.zib.de/doc-6.0.0/html/CMAKE.php, then installed the PySCIPOpt package from source using the instructions found here: https://github.com/SCIP-Interfaces/PySCIPOpt/blob/master/INSTALL.rst. When I executed “from pyscipopt import Model” within Python, I got this error:

Doing “whereis libscip.so” told me that there was a libscip.so file in /usr/lib. After a bit of head scratching, I ran ldd on libscip.so, and found out that it doesn’t provide any symbols. I installed the scipoptsuite from the Arch Linux AUR (see: https://web.archive.org/web/20181014202845/https://aur.archlinux.org/packages/scipoptsuite/). So, I removed the package using pacman, and re-installed PySCIPOpt. Then, running the same command in Python, I got this error:

ImportError: libscip.so.4.0: cannot open shared object file: No such file or directory

Okay, this is a bit more manageable — I’m simply missing a file. After more head-scratching, I grepped for “libscip.so.4.0” within the PySCIPOpt github repo (on my local machine, so in ~/github/PySCIPOpt). This turned up build/lib.linux-x86_64-3.7/pyscipopt/scip.cpython-37m-x86_64-linux-gnu.so

I ran ldd on this file, and got the following:

Well, that explains it. Libscip.so was installed in /usr/local/lib:

One thing I tried was running:

export SCIPOPTDIR=/usr/local/lib

before running “sudo python setup.py install”, but this didn’t fix the problem. But I did learn something very interesting: apparently, exported variables like these aren’t preserved in sudo runs; to get them to stick I have to use the -E option:

sudo -E python setup.py install

I ended up just copying the so file:

cp libscip.so.6.0 /usr/lib/libscip.so.4.0

I tried playing around with symlinks, but they didn’t work too well. Then, I re-built and re-installed PySCIPOpt, and ran ldd on the so file:

And sure enough, it worked! Now, I don’t seem to have any issues with PySCIPOpt

# Rapid Exception Definitions in Python

This is a neat little trick I figured out today. I’m working on a tool for creating MS/MS immunopeptidomics pipelines (see: https://github.com/mrForce/tidePipeline). In it, there’s a file called “Errors.py”, which defines a whole bunch of errors. Many of these errors share the following structure: the constructor takes in a single argument, and the __repr__ function spits out: “prefix message” + argument + ” suffix message”. Like this:

class MSGF2PinFailedError(Error):
def __init__(self, command):
self.command = command
def __repr__(self):
return 'The MSGF2Pin call failed: ' + self.command

First, we’ll define a base class:

class CustomError(Exception):
def __init__(self, thing):
self.thing = thing
def __repr__(self):
return self.prefix + self.thing + self.suffix
def __str__(self):
return self.__repr__()

We can use the type function to easily create subclasses of CustomError in a single line. Let’s add a factory method to the CustomError class to create these subclasses using type:

class CustomError(Exception):
def __init__(self, thing):
self.thing = thing
def __repr__(self):
return self.prefix + self.thing + self.suffix
def __str__(self):
return self.__repr__()
@staticmethod
def factory(name, prefix, suffix):
return type(name, (CustomError,), {'prefix': prefix, 'suffix': suffix})

If we put the following into a script:

class CustomError(Exception):
def __init__(self, thing):
self.thing = thing
def __repr__(self):
return self.prefix + self.thing + self.suffix
def __str__(self):
return self.__repr__()
@staticmethod
def factory(name, prefix, suffix):
return type(name, (CustomError,), {'prefix': prefix, 'suffix': suffix})

JordanError = CustomError.factory('JordanError', 'Hello Jordan ', '')
raise JordanError('Something fucked up')

And we run it, we get:

# Installing KNIME on Arch Linux

I’ve been trying to get KNIME to play nice with Python 3.6 on Arch Linux. So far, I’ve installed Anaconda using the package in the AUR (see: https://aur.archlinux.org/packages/anaconda/), and followed the directions given here: https://www.knime.com/blog/setting-up-the-knime-python-extension-revisited-for-python-30-and-20

I created the Anaconda environment using the following command:

conda create -y -n py36_knime python=3.6 pandas jedi

Then, I created the following script called py36.sh in my home directory:

#! /bin/bash
# start by making sure that the anaconda directory is on the PATH
# so that the source activate command works.
# This isn't necessary if you already know that
# the anaconda bin dir is on the PATH
export PATH="/opt/anaconda/bin:$PATH" source activate py36_knime python "$@" 1>&1 2>&2

Then, to install Google Protobuf, I ran the export and source commands (the same ones used in the py36.sh script), and then:

conda install protobuf

If I run Python, and import google and google.protobuf, it appears that protobuf is installed:

(py36_knime) [jforce@jforce ~]$python Python 3.6.3 |Anaconda, Inc.| (default, Nov 20 2017, 20:41:42) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import google >>> from google import protobuf >>> protobuf <module 'google.protobuf' from '/home/jforce/.conda/envs/py36_knime/lib/python3.6/site-packages/google/protobuf/__init__.py'> However, when I try to apply the settings in KNIME, I get the following error: To see what exactly was going on, I modified my py36.sh script to look like this: #! /bin/bash # start by making sure that the anaconda directory is on the PATH # so that the source activate command works. # This isn't necessary if you already know that # the anaconda bin dir is on the PATH export PATH="/opt/anaconda/bin:$PATH"
echo "zero: $0 one:$1 two: $2" >> args.txt source activate py36_knime python "$@" 1>&1 2>&2

So, it writes the arguments to the py36.sh script to a file called args.txt. After saving, and pressing “Apply” in the KNIME dialog again, args.txt looks like this:

zero: /home/jforce/py36.sh one: /usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py two:

Then, if I open up /usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py in emacs, it tries to import table_pb2 as a way of checking the version of Protobuf:

I’ve managed to figure out that str(e) is “expected bytes, str found”.

I’d like to see where table_pb2 comes from, but googling it doesn’t pull up any clear answers. So, I modified the PythonKernelTester.py script to print to screen whenever it imports a library/class:

# -*- coding: utf-8 -*-

import sys

min_pandas_version = '0.7.0'
min_python_version = '2.5.0'
min_protobuf_version = '2.5.0'

_message = ''

# check libs, output info and exit with 1 if an error occurred
def main():
pythonVersion = sys.version_info
print('Python version: ' + str(pythonVersion[0]) + '.' + str(pythonVersion[1]) + '.' + str(pythonVersion[2]))
check_required_libs()
print(_message)

# check for all libs that are required by the python kernel
def check_required_libs():
python3 = sys.version_info >= (3, 0)
check_version_python()
# these libs should be standard
if python3:
check_lib('io')
else:
check_lib('StringIO')
check_lib('datetime', ['datetime'])
check_lib('math')
check_lib('socket')
check_lib('struct')
check_lib('base64')
check_lib('traceback')
check_lib('os')
check_lib('pickle')
check_lib('imp')
check_lib('types')
# these libs are non standard requirements
check_lib('numpy')
if check_lib('pandas', ['DataFrame'], min_pandas_version):
check_version_pandas()
check_version_protobuf()

def check_lib(lib, cls=[], version=None):
error = False
if not lib_available(lib):
error = True
message = 'Library ' + lib + ' is missing'
if version is not None:
message += ', required minimum version is ' + version
else:
for cl in cls:
if not class_available(lib, cl):
error = True
add_to_message('Class ' + cl + ' in library ' + lib + ' is missing')
return not error

def class_available(lib, cls):
local_env = {}
print('from ' + lib + ' import ' + cls)
exec('try:\n\tfrom ' + lib + ' import ' + cls + '\n\tsuccess = True\nexcept:\n\tsuccess = False', {}, local_env)
return local_env['success']

# returns true if the given library can successfully be imported, false otherwise
def lib_available(lib):
local_env = {}
print('import ' + lib)
exec('try:\n\timport ' + lib + '\n\tsuccess = True\nexcept:\n\tsuccess = False', {}, local_env)
return local_env['success']

def check_version_python():
min_version = min_python_version.split('.')
version = sys.version_info
smaller = False
bigger = False
for i in range(len(min_version)):
if int(version[i]) > int(min_version[i]):
bigger = True
break
if not bigger and int(version[i]) < int(min_version[i]):
smaller = True
break
if smaller:
add_to_message('Installed python version is ' + str(version[0]) + '.' + str(version[1]) + '.' + str(version[2]) + ', required minimum is ' + '.'.join(min_version))

def check_version_pandas():
min_version = min_pandas_version.split('.')
try:
import pandas
version = pandas.__version__.split('.')
if len(version) is not len(min_version):
raise Exception()
smaller = False
bigger = False
for i in range(len(min_version)):
if int(version[i]) > int(min_version[i]):
bigger = True
break
if not bigger and int(version[i]) < int(min_version[i]):
smaller = True
break
if smaller:
add_to_message('Installed pandas version is ' + '.'.join(version) + ', required minimum is ' + '.'.join(min_version))
except:
add_to_message('Could not detect pandas version, required minimum is ' + '.'.join(min_version))

def check_version_protobuf():
try:
import table_pb2
except ModuleNotFoundError as e:
a = str(e)
print('exception: ' + a)
add_to_message('Error while trying to load protobuf:\n' + a + '\nThe minimum required protobuf version is ' + min_protobuf_version)

global _message
_message += line + '\n'

main()

The output looked like this:

(py36_knime) [jforce@jforce ~]$python /usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py Python version: 3.6.3 import io import datetime from datetime import datetime import math import socket import struct import base64 import traceback import os import pickle import imp import types import numpy import pandas from pandas import DataFrame import google.protobuf exception: expected bytes, str found Error while trying to load protobuf: expected bytes, str found The minimum required protobuf version is 2.5.0 If I go into Python, and do these imports, and then try to do “import table_pb2”, I get a ModuleNotFoundError: (py36_knime) [jforce@jforce ~]$ python
Python 3.6.3 |Anaconda, Inc.| (default, Nov 20 2017, 20:41:42)
[GCC 7.2.0] on linux
>>> import io
>>> import datetime
>>> from datetime import datetime
>>> import math
>>> import socket
>>> import struct
>>> import base64
>>> import traceback
>>> import os
>>> import pickle
>>> import imp
>>> import types
>>> import numpy
>>> import pandas
>>> from pandas import DataFrame
>>> import table_pb2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'table_pb2'

This makes sense — it can’t find the table_pb2 module. However, if I change the “except Exception as e” in check_version_protobuf to “except ModuleNotFoundError as e” and re-run the script, I get the following:

(py36_knime) [jforce@jforce ~]$python /usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py Python version: 3.6.3 import io import datetime from datetime import datetime import math import socket import struct import base64 import traceback import os import pickle import imp import types import numpy import pandas from pandas import DataFrame import google.protobuf Traceback (most recent call last): File "/usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py", line 129, in <module> main() File "/usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py", line 16, in main check_required_libs() File "/usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py", line 44, in check_required_libs check_version_protobuf() File "/usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py", line 117, in check_version_protobuf import table_pb2 File "/usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/table_pb2.py", line 16, in <module> serialized_pb='\n\x0btable.proto\x12\x05knime\"\x9f\x18\n\x05Table\x12\r\n\x05valid\x18\x13 \x02(\x08\x12\r\n\x05\x65rror\x18\x14 \x01(\t\x12\x0f\n\x07numCols\x18\x01 \x02(\x05\x12\x0f\n\x07numRows\x18\x02 \x02(\x05\x12\r\n\x05rowId\x18\x03 \x03(\t\x12,\n\x06\x63olRef\x18\x04 \x03(\x0b\x32\x1c.knime.Table.ColumnReference\x12.\n\nbooleanCol\x18\x05 \x03(\x0b\x32\x1a.knime.Table.BooleanColumn\x12\x36\n\x0e\x62ooleanListCol\x18\x06 \x03(\x0b\x32\x1e.knime.Table.BooleanListColumn\x12.\n\nintegerCol\x18\x07 \x03(\x0b\x32\x1a.knime.Table.IntegerColumn\x12\x36\n\x0eintegerListCol\x18\x08 \x03(\x0b\x32\x1e.knime.Table.IntegerListColumn\x12(\n\x07longCol\x18\t \x03(\x0b\x32\x17.knime.Table.LongColumn\x12\x30\n\x0blongListCol\x18\n \x03(\x0b\x32\x1b.knime.Table.LongListColumn\x12,\n\tdoubleCol\x18\x0b \x03(\x0b\x32\x19.knime.Table.DoubleColumn\x12\x34\n\rdoubleListCol\x18\x0c \x03(\x0b\x32\x1d.knime.Table.DoubleListColumn\x12,\n\tstringCol\x18\r \x03(\x0b\x32\x19.knime.Table.StringColumn\x12\x34\n\rstringListCol\x18\x0e \x03(\x0b\x32\x1d.knime.Table.StringListColumn\x12\x36\n\x0e\x64\x61teAndTimeCol\x18\x0f \x03(\x0b\x32\x1e.knime.Table.DateAndTimeColumn\x12>\n\x12\x64\x61teAndTimeListCol\x18\x10 \x03(\x0b\x32\".knime.Table.DateAndTimeListColumn\x12,\n\tobjectCol\x18\x11 \x03(\x0b\x32\x19.knime.Table.ObjectColumn\x12\x34\n\robjectListCol\x18\x12 \x03(\x0b\x32\x1d.knime.Table.ObjectListColumn\x1a\x34\n\x0f\x43olumnReference\x12\x0c\n\x04type\x18\x01 \x02(\t\x12\x13\n\x0bindexInType\x18\x02 \x02(\x05\x1aN\n\rBooleanColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12/\n\x0c\x62ooleanValue\x18\x02 \x03(\x0b\x32\x19.knime.Table.BooleanValue\x1a\x1d\n\x0c\x42ooleanValue\x12\r\n\x05value\x18\x01 \x01(\x08\x1ai\n\x11\x42ooleanListColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\r\n\x05isSet\x18\x02 \x02(\x08\x12\x37\n\x10\x62ooleanListValue\x18\x03 \x03(\x0b\x32\x1d.knime.Table.BooleanListValue\x1aO\n\x10\x42ooleanListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12(\n\x05value\x18\x02 \x03(\x0b\x32\x19.knime.Table.BooleanValue\x1aN\n\rIntegerColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12/\n\x0cintegerValue\x18\x02 \x03(\x0b\x32\x19.knime.Table.IntegerValue\x1a\x1d\n\x0cIntegerValue\x12\r\n\x05value\x18\x01 \x01(\x05\x1ai\n\x11IntegerListColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\r\n\x05isSet\x18\x02 \x02(\x08\x12\x37\n\x10integerListValue\x18\x03 \x03(\x0b\x32\x1d.knime.Table.IntegerListValue\x1aO\n\x10IntegerListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12(\n\x05value\x18\x02 \x03(\x0b\x32\x19.knime.Table.IntegerValue\x1a\x45\n\nLongColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12)\n\tlongValue\x18\x02 \x03(\x0b\x32\x16.knime.Table.LongValue\x1a\x1a\n\tLongValue\x12\r\n\x05value\x18\x01 \x01(\x03\x1a`\n\x0eLongListColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\r\n\x05isSet\x18\x02 \x02(\x08\x12\x31\n\rlongListValue\x18\x03 \x03(\x0b\x32\x1a.knime.Table.LongListValue\x1aI\n\rLongListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12%\n\x05value\x18\x02 \x03(\x0b\x32\x16.knime.Table.LongValue\x1aK\n\x0c\x44oubleColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12-\n\x0b\x64oubleValue\x18\x02 \x03(\x0b\x32\x18.knime.Table.DoubleValue\x1a\x1c\n\x0b\x44oubleValue\x12\r\n\x05value\x18\x01 \x01(\x01\x1a\x66\n\x10\x44oubleListColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\r\n\x05isSet\x18\x02 \x02(\x08\x12\x35\n\x0f\x64oubleListValue\x18\x03 \x03(\x0b\x32\x1c.knime.Table.DoubleListValue\x1aM\n\x0f\x44oubleListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12\'\n\x05value\x18\x02 \x03(\x0b\x32\x18.knime.Table.DoubleValue\x1aK\n\x0cStringColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12-\n\x0bstringValue\x18\x02 \x03(\x0b\x32\x18.knime.Table.StringValue\x1a\x1c\n\x0bStringValue\x12\r\n\x05value\x18\x01 \x01(\t\x1a\x66\n\x10StringListColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\r\n\x05isSet\x18\x02 \x02(\x08\x12\x35\n\x0fstringListValue\x18\x03 \x03(\x0b\x32\x1c.knime.Table.StringListValue\x1aM\n\x0fStringListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12\'\n\x05value\x18\x02 \x03(\x0b\x32\x18.knime.Table.StringValue\x1aZ\n\x11\x44\x61teAndTimeColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\x37\n\x10\x64\x61teAndTimeValue\x18\x02 \x03(\x0b\x32\x1d.knime.Table.DateAndTimeValue\x1a\x7f\n\x10\x44\x61teAndTimeValue\x12\x0c\n\x04year\x18\x01 \x01(\x05\x12\r\n\x05month\x18\x02 \x01(\x05\x12\x0b\n\x03\x64\x61y\x18\x03 \x01(\x05\x12\x0c\n\x04hour\x18\x04 \x01(\x05\x12\x0e\n\x06minute\x18\x05 \x01(\x05\x12\x0e\n\x06second\x18\x06 \x01(\x05\x12\x13\n\x0bmillisecond\x18\x07 \x01(\x05\x1au\n\x15\x44\x61teAndTimeListColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\r\n\x05isSet\x18\x02 \x02(\x08\x12?\n\x14\x64\x61teAndTimeListValue\x18\x03 \x03(\x0b\x32!.knime.Table.DateAndTimeListValue\x1aW\n\x14\x44\x61teAndTimeListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12,\n\x05value\x18\x02 \x03(\x0b\x32\x1d.knime.Table.DateAndTimeValue\x1aY\n\x0cObjectColumn\x12\x0c\n\x04type\x18\x01 \x02(\t\x12\x0c\n\x04name\x18\x02 \x02(\t\x12-\n\x0bobjectValue\x18\x03 \x03(\x0b\x32\x18.knime.Table.ObjectValue\x1a\x1c\n\x0bObjectValue\x12\r\n\x05value\x18\x01 \x01(\x0c\x1at\n\x10ObjectListColumn\x12\x0c\n\x04type\x18\x01 \x02(\t\x12\x0c\n\x04name\x18\x02 \x02(\t\x12\r\n\x05isSet\x18\x03 \x02(\x08\x12\x35\n\x0fobjectListValue\x18\x04 \x03(\x0b\x32\x1c.knime.Table.ObjectListValue\x1aM\n\x0fObjectListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12\'\n\x05value\x18\x02 \x03(\x0b\x32\x18.knime.Table.ObjectValueB3\n\x1dorg.knime.python.kernel.protoB\x12ProtobufKnimeTable') File "/home/jforce/.conda/envs/py36_knime/lib/python3.6/site-packages/google/protobuf/descriptor.py", line 829, in __new__ return _message.default_pool.AddSerializedFile(serialized_pb) TypeError: expected bytes, str found Edit: I’m ending this post now, because I realized that the AUR package is out of date. I’m going to try downloading from KNIME. # IntelliJ IDEA with DWM I use a slightly customized version of the DWM window manager (see this: https://github.com/mrForce/dwm_configuration. When I change which window is in focus, using the keyboard, my mouse follows — and when I re-focus on a previously focused window, the mouse gets placed where it was when it left that window), and IntelliJ wasn’t playing nice with it. I didn’t take a screenshot, but I started IntelliJ IDEA, a grey window would appear. Apparently this is a problem with DWM, and other non-reparenting window managers, as per the DWM manpage: Anyways, I found that installing wmname using pacman, and adding the following 3 lines to my xinitrc made IntelliJ work: export _JAVA_AWT_WM_NONREPARENTING=1 export AWT_TOOLKIT=MToolkit wmname LG3D # Compiling KGEM I’m trying to get KGEM to compile and run (see: http://alan.cs.gsu.edu/NGS/?q=content/kgem). It uses maven for compiling, and relies on biojava; the maven repo for biojava seems to be down. First, you’ll need to clone the KGEM repo: [jforce@jforce kgem_two]$ git clone https://github.com/dr-artio/KGEM.git
Cloning into 'KGEM'...
remote: Counting objects: 1037, done.
remote: Total 1037 (delta 0), reused 0 (delta 0), pack-reused 1037
Receiving objects: 100% (1037/1037), 154.90 KiB | 1.87 MiB/s, done.
Resolving deltas: 100% (393/393), done.
[jforce@jforce kgem_two]$ls KGEM If you go into the KGEM directory, you should see a README file, pom.xml file and src folder. If you download the KGEM.jar file (for an old version) from http://alan.cs.gsu.edu/kgem/KGEM-0.3.1.jar, and open it up, you’ll see something like this: We’ll go into the KGEM folder, and try to build it: [jforce@jforce KGEM]$ mvn clean package

Eventually, we’ll run into the following error:

We’ll have to install  biojava3-core.jar:3.0.5 manually, since the maven repo seems to be down. I found the release here: https://github.com/biojava/biojava/releases/tag/biojava-3.0.5.

We’ll clone the biojava repo first:

[jforce@jforce kgem_two]$git clone https://github.com/biojava/biojava.git Cloning into 'biojava'... remote: Counting objects: 89362, done. remote: Total 89362 (delta 0), reused 0 (delta 0), pack-reused 89362 Receiving objects: 100% (89362/89362), 44.99 MiB | 1.31 MiB/s, done. Resolving deltas: 100% (41743/41743), done. cd into the biojava folder, and checkout the tag for versiot 3.0.5: [jforce@jforce biojava]$ git checkout tags/biojava-3.0.5
Note: checking out 'tags/biojava-3.0.5'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b <new-branch-name>

HEAD is now at 7b3b8871a... [maven-release-plugin] copy for tag biojava-3.0.5

Then, run:

mvn install -rf :biojava3-core

It should build and install the biojava3-core component locally, but fail at installing biojava3-phylo. This is okay, since KGEM only relies on biojava3-core:

Now, cd back into the KGEM directory. Run:

mvn assembly:assembly -P pack

We include the “-P pack” part in the mvn command, since the pack profile in the pom.xml file specifies a main class, and tells maven to also package up the dependencies to create a runnable jar file:

There is a runnable jar file called kgem-0.7.1-jar-with-dependencies.jar in target: