Grading Programming Exams part 2

(this follows from my last post on this subject)

Now, we’ll formulate this as an LP. Then, I’ll write a Python script to solve it, PySCIPOpt as the solver (and maybe I’ll add support for the Scipy LP solver later).

Inputs:

  • Integer N, the number of students
  • Integer n, the number of criteria
  • Target grades \{t_1, t_2, \cdots, t_N\}
  • Criteria scores \{c^{j, i}_1, c^{j, i}_2, \cdots, c^{j, i}_n for each student j
  • Bounds on a subset of the weights: w^{min}_{i} \leq w_{i} \leq w^{max}_{i}

Constraints

  • \sum_{i} c^{max}_{i}w_{i}  = 100
  • For weights with no specified bounds: w_{i} \geq 0
  • For weights with specified bounds: w^{min}_{i} \leq w_{i} \leq w^{max}_{i}
  • For student j: 0 \leq \sum_{i} w_{i}c_{j, i} \leq 100

Objective

Minimize \sum_{j}^{N} \Big( t_{j} - \sum_{i}^{n} w_{i}c_{j, i}\Big)^{2}

Advertisements

Grading Programming Exams

Introduction

I’m a TA for a class on Systems Programming (taught in C). One of the TA responsibilities is to grade the exams. Usually, we will each take a question from the exam, and grade every student for that question. However, this task, especially when you have hundreds of students, is pretty laborious.

It would be nice if I could just give binary grades: either the program works, or it doesn’t. Unfortunately, that wouldn’t be a good reflection of the student’s knowledge, especially on a timed exam: the students who are 90% percent of the way there would get the same grade as the students who barely attempted the problem. Given this, we have to give partial credit based on the quality of the solutions, even if the solution isn’t perfect, or doesn’t compile. In my view, there are two obvious ways of doing this:

  1. Simply look at their solutions, and make a judgment about quality (A, B, C, D, or F).
  2. Have a list of independent, $\latex m_{i}$-ary criteria \{c_1, c_2, \cdots, c_n\} with a set of weights \{w_1, w_2, \cdots, w_n\} such that the student’s grade is \sum_{i = 0}^{n} w_i c_i and the sum of weights is 100. When I say m_{i}-ary, I mean that each criteria c_i can take on a set of m_{i} values. There should be a clear and consistent criteria for deciding the value of c_i. Then, there will also be a set of criteria that don’t have a numerical weight, but is a  necessary condition for a certain grade. For example, to get a 100% on a question, the program written must function perfectly.

Of course, the problem with #1 is that it’s difficult to be consistent across exams; this is partially due to the fact that the way I judge the solutions will probably change as I grade the exam, causing a drift in grades.

The problem with #2 is that it isn’t obvious how to come up with the weights, especially before I start grading the exam. As I grade the exam, I will often start with a grading criteria, with weights, and adjust the weights/remove criteria as I notice patterns in the exam. For example, I might start with having one of the criteria be that the program runs and produces the right output (this would be binary), and have that worth 30% of the question. But, if almost no one is able to achieve this, it might make more sense to say that the program working perfectly is a necessary condition of getting 100%, and drop it as a weighted criteria.

In some cases, I may realize that I need to add a criteria, in which case I need to examine the questions I’ve already graded. This is obviously a hassle, and very time consuming.

But, I think I’ve figured out a way of combining #1 and #2 to create a better grading system.

The System

First, I’ll lay out a grading criteria without pre-specified weights. Each criteria c_i needs to be independent of other criteria, m_i-ary, and consistently grade-able.  Then, I will go through each student’s solution s_j, and assign it a target grade t_j (100, 95, 90, 85, 80, 75, 70, \cdots,  0).

Then, I’ll throw out the extremes: those that get 100%, and those that get 0s. Then, the task is to assign a weight to each criteria that minimizes mean-squared error. This is basically regression, except we say that each weight must be non-negative, the sum of weights must be 100, and some of the weights can have upper and/or lower bounds.

Also, it would be good to have the weights be integers (though that isn’t strictly necessary).

 

I can formulate this as an LP. But that’s for another blog post!

 

Libscip and PySCIPOpt

(NOTE: I’m reconstructing the order of events in how I fixed this, so there might be some fuzzy details)

I was trying to install PySCIPOpt and the SCIP Optimization Suite, to do some homework with.

I followed the installation instructions on http://scip.zib.de/doc-6.0.0/html/CMAKE.php, then installed the PySCIPOpt package from source using the instructions found here: https://github.com/SCIP-Interfaces/PySCIPOpt/blob/master/INSTALL.rst. When I executed “from pyscipopt import Model” within Python, I got this error:

ImportError: /usr/lib/python3.7/site-packages/pyscipopt/scip.cpython-37m-x86_64-linux-gnu.so: undefined symbol: SCIPsolveBendersSubproblem

Doing “whereis libscip.so” told me that there was a libscip.so file in /usr/lib. After a bit of head scratching, I ran ldd on libscip.so, and found out that it doesn’t provide any symbols. I installed the scipoptsuite from the Arch Linux AUR (see: https://web.archive.org/web/20181014202845/https://aur.archlinux.org/packages/scipoptsuite/). So, I removed the package using pacman, and re-installed PySCIPOpt. Then, running the same command in Python, I got this error:

ImportError: libscip.so.4.0: cannot open shared object file: No such file or directory

Okay, this is a bit more manageable — I’m simply missing a file. After more head-scratching, I grepped for “libscip.so.4.0” within the PySCIPOpt github repo (on my local machine, so in ~/github/PySCIPOpt). This turned up build/lib.linux-x86_64-3.7/pyscipopt/scip.cpython-37m-x86_64-linux-gnu.so

I ran ldd on this file, and got the following:

2018-10-14-163443_948x200_scrot

Well, that explains it. Libscip.so was installed in /usr/local/lib:

2018-10-14-163758_860x90_scrot

One thing I tried was running:

export SCIPOPTDIR=/usr/local/lib

before running “sudo python setup.py install”, but this didn’t fix the problem. But I did learn something very interesting: apparently, exported variables like these aren’t preserved in sudo runs; to get them to stick I have to use the -E option:

 

sudo -E python setup.py install

I ended up just copying the so file:

 

cp libscip.so.6.0 /usr/lib/libscip.so.4.0

I tried playing around with symlinks, but they didn’t work too well. Then, I re-built and re-installed PySCIPOpt, and ran ldd on the so file:

 

2018-10-14-164511_932x387_scrot

 

And sure enough, it worked! Now, I don’t seem to have any issues with PySCIPOpt

Rapid Exception Definitions in Python

This is a neat little trick I figured out today. I’m working on a tool for creating MS/MS immunopeptidomics pipelines (see: https://github.com/mrForce/tidePipeline). In it, there’s a file called “Errors.py”, which defines a whole bunch of errors. Many of these errors share the following structure: the constructor takes in a single argument, and the __repr__ function spits out: “prefix message” + argument + ” suffix message”. Like this:


class MSGF2PinFailedError(Error):
  def __init__(self, command):
    self.command = command
  def __repr__(self):
    return 'The MSGF2Pin call failed: ' + self.command

First, we’ll define a base class:


class CustomError(Exception):
  def __init__(self, thing):
    self.thing = thing
  def __repr__(self):
    return self.prefix + self.thing + self.suffix
  def __str__(self):
    return self.__repr__()

We can use the type function to easily create subclasses of CustomError in a single line. Let’s add a factory method to the CustomError class to create these subclasses using type:


class CustomError(Exception):
  def __init__(self, thing):
    self.thing = thing
  def __repr__(self):
    return self.prefix + self.thing + self.suffix
  def __str__(self):
    return self.__repr__()
  @staticmethod
  def factory(name, prefix, suffix):
    return type(name, (CustomError,), {'prefix': prefix, 'suffix': suffix})

If we put the following into a script:

class CustomError(Exception):
    def __init__(self, thing):
        self.thing = thing
    def __repr__(self):
        return self.prefix + self.thing + self.suffix
    def __str__(self):
        return self.__repr__()
    @staticmethod
    def factory(name, prefix, suffix):
        return type(name, (CustomError,), {'prefix': prefix, 'suffix': suffix})                                                                             

JordanError = CustomError.factory('JordanError', 'Hello Jordan ', '')
raise JordanError('Something fucked up')

And we run it, we get:

 

2018-08-08-180555_513x92_scrot.png

 

Installing KNIME on Arch Linux

I’ve been trying to get KNIME to play nice with Python 3.6 on Arch Linux. So far, I’ve installed Anaconda using the package in the AUR (see: https://aur.archlinux.org/packages/anaconda/), and followed the directions given here: https://www.knime.com/blog/setting-up-the-knime-python-extension-revisited-for-python-30-and-20

I created the Anaconda environment using the following command:

 

conda create -y -n py36_knime python=3.6 pandas jedi

 

Then, I created the following script called py36.sh in my home directory:

#! /bin/bash 
# start by making sure that the anaconda directory is on the PATH 
# so that the source activate command works. 
# This isn't necessary if you already know that 
# the anaconda bin dir is on the PATH 
export PATH="/opt/anaconda/bin:$PATH" 

source activate py36_knime 
python "$@" 1>&1 2>&2

 

Then, to install Google Protobuf, I ran the export and source commands (the same ones used in the py36.sh script), and then:

conda install protobuf

If I run Python, and import google and google.protobuf, it appears that protobuf is installed:

 

(py36_knime) [jforce@jforce ~]$ python
Python 3.6.3 |Anaconda, Inc.| (default, Nov 20 2017, 20:41:42) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
>>> from google import protobuf
>>> protobuf
<module 'google.protobuf' from '/home/jforce/.conda/envs/py36_knime/lib/python3.6/site-packages/google/protobuf/__init__.py'>

 

However, when I try to apply the settings in KNIME, I get the following error:

2017-12-21-120654_1366x768_scrot.png

To see what exactly was going on, I modified my py36.sh script to look like this:

#! /bin/bash 
# start by making sure that the anaconda directory is on the PATH 
# so that the source activate command works. 
# This isn't necessary if you already know that 
# the anaconda bin dir is on the PATH 
export PATH="/opt/anaconda/bin:$PATH" 
echo "zero: $0 one: $1 two: $2" >> args.txt 
source activate py36_knime 
python "$@" 1>&1 2>&2

So, it writes the arguments to the py36.sh script to a file called args.txt. After saving, and pressing “Apply” in the KNIME dialog again, args.txt looks like this:

 

zero: /home/jforce/py36.sh one: /usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py two:

 

Then, if I open up /usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py in emacs, it tries to import table_pb2 as a way of checking the version of Protobuf:

 

2017-12-21-121658_1366x768_scrot

 

I’ve managed to figure out that str(e) is “expected bytes, str found”.

I’d like to see where table_pb2 comes from, but googling it doesn’t pull up any clear answers. So, I modified the PythonKernelTester.py script to print to screen whenever it imports a library/class:

# -*- coding: utf-8 -*-

import sys

min_pandas_version = '0.7.0'
min_python_version = '2.5.0'
min_protobuf_version = '2.5.0'

_message = ''




# check libs, output info and exit with 1 if an error occurred
def main():
 pythonVersion = sys.version_info
 print('Python version: ' + str(pythonVersion[0]) + '.' + str(pythonVersion[1]) + '.' + str(pythonVersion[2]))
 check_required_libs()
 print(_message)




# check for all libs that are required by the python kernel
def check_required_libs():
 python3 = sys.version_info >= (3, 0)
 check_version_python()
 # these libs should be standard
 if python3:
 check_lib('io')
 else:
 check_lib('StringIO')
 check_lib('datetime', ['datetime'])
 check_lib('math')
 check_lib('socket')
 check_lib('struct')
 check_lib('base64')
 check_lib('traceback')
 check_lib('os')
 check_lib('pickle')
 check_lib('imp')
 check_lib('types')
 # these libs are non standard requirements
 check_lib('numpy')
 if check_lib('pandas', ['DataFrame'], min_pandas_version):
 check_version_pandas()
 if check_lib('google.protobuf', version=min_protobuf_version):
 check_version_protobuf()




def check_lib(lib, cls=[], version=None):
 error = False
 if not lib_available(lib):
 error = True
 message = 'Library ' + lib + ' is missing'
 if version is not None:
 message += ', required minimum version is ' + version
 add_to_message(message)
 else:
 for cl in cls:
 if not class_available(lib, cl):
 error = True
 add_to_message('Class ' + cl + ' in library ' + lib + ' is missing')
 return not error




def class_available(lib, cls):
 local_env = {}
 print('from ' + lib + ' import ' + cls)
 exec('try:\n\tfrom ' + lib + ' import ' + cls + '\n\tsuccess = True\nexcept:\n\tsuccess = False', {}, local_env)
 return local_env['success']

# returns true if the given library can successfully be imported, false otherwise
def lib_available(lib):
 local_env = {}
 print('import ' + lib)
 exec('try:\n\timport ' + lib + '\n\tsuccess = True\nexcept:\n\tsuccess = False', {}, local_env)
 return local_env['success']




def check_version_python():
 min_version = min_python_version.split('.')
 version = sys.version_info
 smaller = False
 bigger = False
 for i in range(len(min_version)):
 if int(version[i]) > int(min_version[i]):
 bigger = True
 break
 if not bigger and int(version[i]) < int(min_version[i]):
 smaller = True
 break
 if smaller:
 add_to_message('Installed python version is ' + str(version[0]) + '.' + str(version[1]) + '.' + str(version[2]) + ', required minimum is ' + '.'.join(min_version))




def check_version_pandas():
 min_version = min_pandas_version.split('.')
 try:
 import pandas
 version = pandas.__version__.split('.')
 if len(version) is not len(min_version):
 raise Exception()
 smaller = False
 bigger = False
 for i in range(len(min_version)):
 if int(version[i]) > int(min_version[i]):
 bigger = True
 break
 if not bigger and int(version[i]) < int(min_version[i]):
 smaller = True
 break
 if smaller:
 add_to_message('Installed pandas version is ' + '.'.join(version) + ', required minimum is ' + '.'.join(min_version))
 except:
 add_to_message('Could not detect pandas version, required minimum is ' + '.'.join(min_version))




def check_version_protobuf():
 try:
 import table_pb2
 except ModuleNotFoundError as e:
 a = str(e)
 print('exception: ' + a)
 add_to_message('Error while trying to load protobuf:\n' + a + '\nThe minimum required protobuf version is ' + min_protobuf_version)




def add_to_message(line):
 global _message
 _message += line + '\n'




main()

The output looked like this:

(py36_knime) [jforce@jforce ~]$ python /usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py 
Python version: 3.6.3
import io
import datetime
from datetime import datetime
import math
import socket
import struct
import base64
import traceback
import os
import pickle
import imp
import types
import numpy
import pandas
from pandas import DataFrame
import google.protobuf
exception: expected bytes, str found
Error while trying to load protobuf:
expected bytes, str found
The minimum required protobuf version is 2.5.0

 

If I go into Python, and do these imports, and then try to do “import table_pb2”, I get a ModuleNotFoundError:

 

(py36_knime) [jforce@jforce ~]$ python
Python 3.6.3 |Anaconda, Inc.| (default, Nov 20 2017, 20:41:42) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import io
>>> import datetime
>>> from datetime import datetime
>>> import math
>>> import socket
>>> import struct
>>> import base64
>>> import traceback
>>> import os
>>> import pickle
>>> import imp
>>> import types
>>> import numpy
>>> import pandas
>>> from pandas import DataFrame
>>> import google.protobuf
>>> import table_pb2
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'table_pb2'

This makes sense — it can’t find the table_pb2 module. However, if I change the “except Exception as e” in check_version_protobuf to “except ModuleNotFoundError as e” and re-run the script, I get the following:

(py36_knime) [jforce@jforce ~]$ python /usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py 
Python version: 3.6.3
import io
import datetime
from datetime import datetime
import math
import socket
import struct
import base64
import traceback
import os
import pickle
import imp
import types
import numpy
import pandas
from pandas import DataFrame
import google.protobuf
Traceback (most recent call last):
 File "/usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py", line 129, in <module>
 main()
 File "/usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py", line 16, in main
 check_required_libs()
 File "/usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py", line 44, in check_required_libs
 check_version_protobuf()
 File "/usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/PythonKernelTester.py", line 117, in check_version_protobuf
 import table_pb2
 File "/usr/share/java/knime-desktop/plugins/org.knime.python_3.3.0.v201611242050/py/table_pb2.py", line 16, in <module>
 serialized_pb='\n\x0btable.proto\x12\x05knime\"\x9f\x18\n\x05Table\x12\r\n\x05valid\x18\x13 \x02(\x08\x12\r\n\x05\x65rror\x18\x14 \x01(\t\x12\x0f\n\x07numCols\x18\x01 \x02(\x05\x12\x0f\n\x07numRows\x18\x02 \x02(\x05\x12\r\n\x05rowId\x18\x03 \x03(\t\x12,\n\x06\x63olRef\x18\x04 \x03(\x0b\x32\x1c.knime.Table.ColumnReference\x12.\n\nbooleanCol\x18\x05 \x03(\x0b\x32\x1a.knime.Table.BooleanColumn\x12\x36\n\x0e\x62ooleanListCol\x18\x06 \x03(\x0b\x32\x1e.knime.Table.BooleanListColumn\x12.\n\nintegerCol\x18\x07 \x03(\x0b\x32\x1a.knime.Table.IntegerColumn\x12\x36\n\x0eintegerListCol\x18\x08 \x03(\x0b\x32\x1e.knime.Table.IntegerListColumn\x12(\n\x07longCol\x18\t \x03(\x0b\x32\x17.knime.Table.LongColumn\x12\x30\n\x0blongListCol\x18\n \x03(\x0b\x32\x1b.knime.Table.LongListColumn\x12,\n\tdoubleCol\x18\x0b \x03(\x0b\x32\x19.knime.Table.DoubleColumn\x12\x34\n\rdoubleListCol\x18\x0c \x03(\x0b\x32\x1d.knime.Table.DoubleListColumn\x12,\n\tstringCol\x18\r \x03(\x0b\x32\x19.knime.Table.StringColumn\x12\x34\n\rstringListCol\x18\x0e \x03(\x0b\x32\x1d.knime.Table.StringListColumn\x12\x36\n\x0e\x64\x61teAndTimeCol\x18\x0f \x03(\x0b\x32\x1e.knime.Table.DateAndTimeColumn\x12>\n\x12\x64\x61teAndTimeListCol\x18\x10 \x03(\x0b\x32\".knime.Table.DateAndTimeListColumn\x12,\n\tobjectCol\x18\x11 \x03(\x0b\x32\x19.knime.Table.ObjectColumn\x12\x34\n\robjectListCol\x18\x12 \x03(\x0b\x32\x1d.knime.Table.ObjectListColumn\x1a\x34\n\x0f\x43olumnReference\x12\x0c\n\x04type\x18\x01 \x02(\t\x12\x13\n\x0bindexInType\x18\x02 \x02(\x05\x1aN\n\rBooleanColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12/\n\x0c\x62ooleanValue\x18\x02 \x03(\x0b\x32\x19.knime.Table.BooleanValue\x1a\x1d\n\x0c\x42ooleanValue\x12\r\n\x05value\x18\x01 \x01(\x08\x1ai\n\x11\x42ooleanListColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\r\n\x05isSet\x18\x02 \x02(\x08\x12\x37\n\x10\x62ooleanListValue\x18\x03 \x03(\x0b\x32\x1d.knime.Table.BooleanListValue\x1aO\n\x10\x42ooleanListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12(\n\x05value\x18\x02 \x03(\x0b\x32\x19.knime.Table.BooleanValue\x1aN\n\rIntegerColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12/\n\x0cintegerValue\x18\x02 \x03(\x0b\x32\x19.knime.Table.IntegerValue\x1a\x1d\n\x0cIntegerValue\x12\r\n\x05value\x18\x01 \x01(\x05\x1ai\n\x11IntegerListColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\r\n\x05isSet\x18\x02 \x02(\x08\x12\x37\n\x10integerListValue\x18\x03 \x03(\x0b\x32\x1d.knime.Table.IntegerListValue\x1aO\n\x10IntegerListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12(\n\x05value\x18\x02 \x03(\x0b\x32\x19.knime.Table.IntegerValue\x1a\x45\n\nLongColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12)\n\tlongValue\x18\x02 \x03(\x0b\x32\x16.knime.Table.LongValue\x1a\x1a\n\tLongValue\x12\r\n\x05value\x18\x01 \x01(\x03\x1a`\n\x0eLongListColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\r\n\x05isSet\x18\x02 \x02(\x08\x12\x31\n\rlongListValue\x18\x03 \x03(\x0b\x32\x1a.knime.Table.LongListValue\x1aI\n\rLongListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12%\n\x05value\x18\x02 \x03(\x0b\x32\x16.knime.Table.LongValue\x1aK\n\x0c\x44oubleColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12-\n\x0b\x64oubleValue\x18\x02 \x03(\x0b\x32\x18.knime.Table.DoubleValue\x1a\x1c\n\x0b\x44oubleValue\x12\r\n\x05value\x18\x01 \x01(\x01\x1a\x66\n\x10\x44oubleListColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\r\n\x05isSet\x18\x02 \x02(\x08\x12\x35\n\x0f\x64oubleListValue\x18\x03 \x03(\x0b\x32\x1c.knime.Table.DoubleListValue\x1aM\n\x0f\x44oubleListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12\'\n\x05value\x18\x02 \x03(\x0b\x32\x18.knime.Table.DoubleValue\x1aK\n\x0cStringColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12-\n\x0bstringValue\x18\x02 \x03(\x0b\x32\x18.knime.Table.StringValue\x1a\x1c\n\x0bStringValue\x12\r\n\x05value\x18\x01 \x01(\t\x1a\x66\n\x10StringListColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\r\n\x05isSet\x18\x02 \x02(\x08\x12\x35\n\x0fstringListValue\x18\x03 \x03(\x0b\x32\x1c.knime.Table.StringListValue\x1aM\n\x0fStringListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12\'\n\x05value\x18\x02 \x03(\x0b\x32\x18.knime.Table.StringValue\x1aZ\n\x11\x44\x61teAndTimeColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\x37\n\x10\x64\x61teAndTimeValue\x18\x02 \x03(\x0b\x32\x1d.knime.Table.DateAndTimeValue\x1a\x7f\n\x10\x44\x61teAndTimeValue\x12\x0c\n\x04year\x18\x01 \x01(\x05\x12\r\n\x05month\x18\x02 \x01(\x05\x12\x0b\n\x03\x64\x61y\x18\x03 \x01(\x05\x12\x0c\n\x04hour\x18\x04 \x01(\x05\x12\x0e\n\x06minute\x18\x05 \x01(\x05\x12\x0e\n\x06second\x18\x06 \x01(\x05\x12\x13\n\x0bmillisecond\x18\x07 \x01(\x05\x1au\n\x15\x44\x61teAndTimeListColumn\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\r\n\x05isSet\x18\x02 \x02(\x08\x12?\n\x14\x64\x61teAndTimeListValue\x18\x03 \x03(\x0b\x32!.knime.Table.DateAndTimeListValue\x1aW\n\x14\x44\x61teAndTimeListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12,\n\x05value\x18\x02 \x03(\x0b\x32\x1d.knime.Table.DateAndTimeValue\x1aY\n\x0cObjectColumn\x12\x0c\n\x04type\x18\x01 \x02(\t\x12\x0c\n\x04name\x18\x02 \x02(\t\x12-\n\x0bobjectValue\x18\x03 \x03(\x0b\x32\x18.knime.Table.ObjectValue\x1a\x1c\n\x0bObjectValue\x12\r\n\x05value\x18\x01 \x01(\x0c\x1at\n\x10ObjectListColumn\x12\x0c\n\x04type\x18\x01 \x02(\t\x12\x0c\n\x04name\x18\x02 \x02(\t\x12\r\n\x05isSet\x18\x03 \x02(\x08\x12\x35\n\x0fobjectListValue\x18\x04 \x03(\x0b\x32\x1c.knime.Table.ObjectListValue\x1aM\n\x0fObjectListValue\x12\x11\n\tisMissing\x18\x01 \x02(\x08\x12\'\n\x05value\x18\x02 \x03(\x0b\x32\x18.knime.Table.ObjectValueB3\n\x1dorg.knime.python.kernel.protoB\x12ProtobufKnimeTable')
 File "/home/jforce/.conda/envs/py36_knime/lib/python3.6/site-packages/google/protobuf/descriptor.py", line 829, in __new__
 return _message.default_pool.AddSerializedFile(serialized_pb)
TypeError: expected bytes, str found

 

Edit: I’m ending this post now, because I realized that the AUR package is out of date. I’m going to try downloading from KNIME.

IntelliJ IDEA with DWM

I use a slightly customized version of the DWM window manager (see this: https://github.com/mrForce/dwm_configuration. When I change which window is in focus, using the keyboard, my mouse follows — and when I re-focus on a previously focused window, the mouse gets placed where it was when it left that window), and IntelliJ wasn’t playing nice with it.

 

I didn’t take a screenshot, but I started IntelliJ IDEA, a grey window would appear. Apparently this is a problem with DWM, and other non-reparenting window managers, as per the DWM manpage:

 

2017-11-29-100431_1278x1003_scrot

 

Anyways, I found that installing wmname using pacman, and adding the following 3 lines to my xinitrc made IntelliJ work:

export _JAVA_AWT_WM_NONREPARENTING=1 
export AWT_TOOLKIT=MToolkit 
wmname LG3D

Compiling KGEM

I’m trying to get KGEM to compile and run (see: http://alan.cs.gsu.edu/NGS/?q=content/kgem). It uses maven for compiling, and relies on biojava; the maven repo for biojava seems to be down.

 

First, you’ll need to clone the KGEM repo:

 

[jforce@jforce kgem_two]$ git clone https://github.com/dr-artio/KGEM.git
Cloning into 'KGEM'...
remote: Counting objects: 1037, done.
remote: Total 1037 (delta 0), reused 0 (delta 0), pack-reused 1037
Receiving objects: 100% (1037/1037), 154.90 KiB | 1.87 MiB/s, done.
Resolving deltas: 100% (393/393), done.
[jforce@jforce kgem_two]$ ls
KGEM

 

If you go into the KGEM directory, you should see a README file, pom.xml file and src folder.

If you download the KGEM.jar file (for an old version) from http://alan.cs.gsu.edu/kgem/KGEM-0.3.1.jar, and open it up, you’ll see something like this:

 

2017-11-28-183936_1278x1003_scrot

 

We’ll go into the KGEM folder, and try to build it:

[jforce@jforce KGEM]$ mvn clean package

 

Eventually, we’ll run into the following error:

 

2017-11-28-185813_1278x1003_scrot

 

We’ll have to install  biojava3-core.jar:3.0.5 manually, since the maven repo seems to be down. I found the release here: https://github.com/biojava/biojava/releases/tag/biojava-3.0.5.

We’ll clone the biojava repo first:

[jforce@jforce kgem_two]$ git clone https://github.com/biojava/biojava.git
Cloning into 'biojava'...
remote: Counting objects: 89362, done.
remote: Total 89362 (delta 0), reused 0 (delta 0), pack-reused 89362
Receiving objects: 100% (89362/89362), 44.99 MiB | 1.31 MiB/s, done.
Resolving deltas: 100% (41743/41743), done.

 

cd into the biojava folder, and checkout the tag for versiot 3.0.5:

 

[jforce@jforce biojava]$ git checkout tags/biojava-3.0.5
Note: checking out 'tags/biojava-3.0.5'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b <new-branch-name>

HEAD is now at 7b3b8871a... [maven-release-plugin] copy for tag biojava-3.0.5



 

Then, run:

mvn install -rf :biojava3-core

 

It should build and install the biojava3-core component locally, but fail at installing biojava3-phylo. This is okay, since KGEM only relies on biojava3-core:

 

2017-11-28-190750_1278x1003_scrot

Now, cd back into the KGEM directory. Run:

 

mvn assembly:assembly -P pack

 

We include the “-P pack” part in the mvn command, since the pack profile in the pom.xml file specifies a main class, and tells maven to also package up the dependencies to create a runnable jar file:

 

2017-11-28-191300_1278x1003_scrot

 

There is a runnable jar file called kgem-0.7.1-jar-with-dependencies.jar in target:

 

2017-11-28-191553_1278x1003_scrot