Highlights of fixes and optimisations to Bb config 2018

-
Originally posted on Nov 8, 2018.

This is a follow up to last years post Highlights of fixes from our upgrade implementation plan where I've tried to summarise all the little fixes that we have had to implement due to issues with the Blackboard application, and which the installer has a nasty habit of removing each time it is used to perform an upgrade.

This is purely for information purposes and Blackboard support should be consulted before making changes to your installation.

Apologies for the occasional formatting issues.

Fixes in bb-config.properties

Files affected:
/usr/local/blackboard/config/bb-config.properties
/usr/local/blackboard/config/bb-config.properties.template

DEV/PREPROD/LIVE Java Log Spam fix

Files affected:
/usr/local/blackboard/config/bb-config.properties

To prevent log spam in the stdout-stderr log perform the following:
#on each node
mkdir -p /home/bbuser/.java/.userPrefs mkdir /home/bbuser/.java/.systemPrefs chown -R bbuser:bbuser /home/bbuser/.java/ chmod 755 -Rv /home/bbuser/.java/ #on each node #within /usr/local/blackboard/config/bb-config.properties #add to bbconfig.jvm.options.extra.tomcat= -Djava.util.prefs.systemRoot=/home/bbuser/.java/
Verification
cat /usr/local/blackboard/config/bb-config.properties |grep Djava
result should contain: bbconfig.jvm.options.extra.tomcat=-Djava.util.prefs.systemRoot=/home/bbuser/.java/
Depending on environment additional entries will be included within this stanza.

DEV/PREPROD/LIVE SCORM connection loss fix

Files affected:
/usr/local/blackboard/config/bb-config.properties
/usr/local/blackboard/config/bb-config.properties.template

As recommended by Blackboard support in case #02455370 apply a fix in two files to prevent SCORM learning objects losing connection to Blackboard.
As bbuser edit bb-config.properties.template on each node and make the following change.
#as bbuser
#on each node
sed -i "s/bbconfig.headers.CSP.frame-ancestors=self/bbconfig.headers.CSP.frame-ancestors=self\' https:\/\/\*.soton.ac.uk \'soton.ac.uk/g" /usr/local/blackboard/config/bb-config.properties.template ;
sed -i "s/bbconfig.headers.CSP.frame-ancestors=self/bbconfig.headers.CSP.frame-ancestors=self\' https:\/\/\*.soton.ac.uk \'soton.ac.uk/g" /usr/local/blackboard/config/bb-config.properties
Verification
#on each node
cat /usr/local/blackboard/config/bb-config.properties.template | grep "ancestors=self'"
#Result should be bbconfig.headers.CSP.frame-ancestors=self' https://*.soton.ac.uk 'soton.ac.uk
#on each node
cat /usr/local/blackboard/config/bb-config.properties.| grep "ancestors=self'"
#Result should be bbconfig.headers.CSP.frame-ancestors=self' https://*.soton.ac.uk 'soton.ac.uk

DEV/PREPROD/LIVE Resolve "JDK 1.8.0_151 and 1.8.0_152 Break Grade Center" issue

Files affected:
/usr/local/blackboard/config/bb-config.properties

According to https://blackboard.secure.force.com/apex/btbb_articleview?id=kA3390000004HdwCAE the Java Development Kits (jdk1.8.0_152 and jdk1.8.0_151) break the Grade Center when compression is turned on, displaying only JSON errors.

To avoid this happening due to an automatic java update we will follow the recommended steps to turn off http compression.
#As bbuser
#On each node
sed -i 's/bbconfig.appserver.http.compression=on/bbconfig.appserver.http.compression=off/' /usr/local/blackboard/config/bb-config.properties
Verification:
#As bbuser
#On each node
cat /usr/local/blackboard/config/bb-config.properties |grep bbconfig.appserver.http.compression
result should be bbconfig.appserver.http.compression=off

Summary of all changes made to bb-config.properties

Files affected:
/usr/local/blackboard/config/bb-config.properties
/usr/local/blackboard/config/bb-config.properties.template

The bb-config.properties file is one of the primary locations for Blackboard settings. With the 2016 Q4 upgrade we made the following adjustments. Note that this includes the Java log spam, http compression and SCORM connection loss fix already detailed above.
As bbuser edit the bb-config.properties file on each node and make the following changes.
#as bbuser
#on each node
nano /usr/local/blackboard/config/bb-config.properties

DEV/PREPROD/LIVE Change 1

#Change bbconfig.max.stacksize.tomcat=
#from 400k to 512k
i.e. bbconfig.max.stacksize.tomcat=512k
Verification
#On each node
#Change 1
cat /usr/local/blackboard/config/bb-config.properties | grep "bbconfig.max.stacksize.tomcat"
#Result should be bbconfig.max.stacksize.tomcat=512k

PREPROD/LIVE Change 2

#Change bbconfig.jvm.options.extra.tomcat=
#So that whole section is replaced with
bbconfig.jvm.options.extra.tomcat=-XX:NewSize=2048m -XX:MaxNewSize=2048m -XX:+UseTLAB -XX:SurvivorRatio=4 -XX:+UseCompressedOops -Xss512k -XX:+PrintVMOptions -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+PrintGCTaskTimeStamps -XX:+PrintCommandLineFlags -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -XX:+AlwaysPreTouch -verbose:gc -XX:+DoEscapeAnalysis -Xverify:none -XX:+UseMontgomeryMultiplyIntrinsic -XX:+UseMontgomerySquareIntrinsic -XX:MetaspaceSize=768m -XX:MaxMetaspaceSize=1536m -Djava.util.prefs.systemRoot=/home/bbuser/.java/
Verification
cat /usr/local/blackboard/config/bb-config.properties |grep "Djava"
#Result should be bbconfig.jvm.options.extra.tomcat=--XX:NewSize=2048m -XX:MaxNewSize=2048m -XX:+UseTLAB -XX:SurvivorRatio=4 -XX:+UseCompressedOops -Xss512k -XX:+PrintVMOptions -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+PrintGCTaskTimeStamps -XX:+PrintCommandLineFlags -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -XX:+AlwaysPreTouch -verbose:gc -XX:+DoEscapeAnalysis -Xverify:none -XX:+UseMontgomeryMultiplyIntrinsic -XX:+UseMontgomerySquareIntrinsic -XX:MetaspaceSize=768m -XX:MaxMetaspaceSize=1536m -Djava.util.prefs.systemRoot=/home/bbuser/.java/

PREPROD/LIVE Change 3

#Change bbconfig.jvm.options.gc=
#So that whole section is replaced with
bbconfig.jvm.options.gc=-XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:InitiatingHeapOccupancyPercent=45 -XX:+UseStringDeduplication
Verification
cat /usr/local/blackboard/config/bb-config.properties |grep UseG1GC
#Result should be
bbconfig.jvm.options.gc=-XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:InitiatingHeapOccupancyPercent=45 -XX:+UseStringDeduplication

DEV/PREPROD/LIVE Change 4

#Check dmarc setting, ensure that: bbconfig.email.use.dmarc.from.override=false
Verification
cat /usr/local/blackboard/config/bb-config.properties | grep "dmarc.from.override"
#Result should be bbconfig.email.use.dmarc.from.override=false

DEV/PREPROD/LIVE Change 5

#Update click jacking setting to resolve issue where
#SCORM learning objects suffer disconnections #Blackboard case #02455370
#Change
#bbconfig.headers.CSP.frame-ancestors=[none OR self OR <comma-delimited URL list>]
bbconfig.headers.CSP.frame-ancestors=self
#to
#bbconfig.headers.CSP.frame-ancestors=[none OR self OR <comma-delimited URL list>]
bbconfig.headers.CSP.frame-ancestors=self' https://*.soton.ac.uk 'soton.ac.uk
#Note that this is the last line the file.
Verification
cat /usr/local/blackboard/config/bb-config.properties | grep "ancestors=self'"
#Result should be bbconfig.headers.CSP.frame-ancestors=self' https://*.soton.ac.uk 'soton.ac.uk

DEV/PREPROD/LIVE Change 6

According to https://blackboard.secure.force.com/apex/btbb_articleview?id=kA3390000004HdwCAE the Java Development Kits (jdk1.8.0_152 and jdk1.8.0_151) break the Grade Center when compression is turned on, displaying only JSON errors.
This is apparently now fixed.
To avoid this happening due to an automatic java update we will follow the recommended steps to turn off http compression.
#As bbuser
#On each node
sed -i 's/bbconfig.appserver.http.compression=on/bbconfig.appserver.http.compression=off/' /usr/local/blackboard/config/bb-config.properties
Verification:
#As bbuser
#On each node
cat /usr/local/blackboard/config/bb-config.properties |grep bbconfig.appserver.http.compression
#result should be bbconfig.appserver.http.compression=off

DEV/PREPROD/LIVE Change 7


Raise bbconfig.cs.database.maxpoolsize from 25 to 50.
#As bbuser
#On each node
sed -i 's/bbconfig.cs.database.maxpoolsize=25/bbconfig.cs.database.maxpoolsize=50/' /usr/local/blackboard/config/bb-config.properties
Verification:
#As bbuser
#On each node
cat /usr/local/blackboard/config/bb-config.properties |grep bbconfig.cs.database.maxpoolsize
#result should be bbconfig.cs.database.maxpoolsize=50

DEV/PREPROD/LIVE Change 8


Set Grade Center caching to zero.
See https://blackboard.secure.force.com/btbb_articleview?id=kA339000000CevB for background.
# as bbuser
# on each node
nano /usr/local/blackboard/config/bb-config.properties
# edit
bbconfig.gradecenter.cache.grade_threshold
# to
bbconfig.gradecenter.cache.grade_threshold=0
Verification
cat /usr/local/blackboard/config/bb-config.properties| grep "bbconfig.gradecenter.cache.grade_threshold=0" |wc -l

Fixes in other files

DEV/PREPROD/LIVE XSS security fixes

This is not needed from 2018 Q2 onwards
Blackboard have recommended adjusting two files in order to resolve XSS issues.
See:
Blackboard files affected:
Dev:/usr/local/blackboard/content/vi/BBLEARN1/plugins/bb-xss-filter/webapp/WEB-INF/classes/blackboard/xss/regex/bb-xss-regex-filter.properties
Preprod/Live: /usr/local/blackboard/content/vi/bb_bb60/plugins/bb-xss-filter/webapp/WEB-INF/classes/blackboard/xss/regex/bb-xss-regex-filter.properties
/usr/local/blackboard/config/internal/bb-file-filter-configuration.properties

Apply fix to bb-xss-regex-filter.properties

All commands are run as bbuser.
For dev environments use /usr/local/blackboard/content/vi/BBLEARN1/plugins/bb-xss-filter/webapp/WEB-INF/classes/blackboard/xss/regex/bb-xss-regex-filter.properties
This is performed on any one node because the file is in a shared folder.
Edit the bb-xss-regex-filter.properties file
nano /usr/local/blackboard/content/vi/bb_bb60/plugins/bb-xss-filter/webapp/WEB-INF/classes/blackboard/xss/regex/bb-xss-regex-filter.properties
Change
disabled_tags.template=(?<=<)\\s*@X@tag@X@(?=[\\s/|>])|(?<=</)\\s*@X@tag@X@(?=\\s*>?)
to
disabled_tags.template=(?<=<)\\s*(?>[^:<>]+:)?\\s*@X@tag@X@(?=[\\s/|>])|(?<=</)\\s*(?>[^:<>]+:)?\\s*@X@tag@X@(?=\\s*>?)

Test change to to bb-xss-regex-filter.properties

Performed by Apps Man. All commands are run as bbuser.
This is performed on on any one node.
Run the command:
cat /usr/local/blackboard/content/vi/bb_bb60/plugins/bb-xss-filter/webapp/WEB-INF/classes/blackboard/xss/regex/bb-xss-regex-filter.properties | grep disabled.tags.template;
Confirm the following is displayed:
# The disabled_tags.template pattern will be applied to each tag in the comma-delimited list disabled_tags.template=(?<=<)\\s*(?>[^:<>]+:)?\\s*@X@tag@X@(?=[\\s/|>])|(?<=</)\\s*(?>[^:<>]+:)?\\s*@X@tag@X@(?=\\s*>?)

Apply fix to bb-file-filter-configuration.properties

All commands are run as bbuser.
This is performed on every node
# On each node
# As bbuser
nano /usr/local/blackboard/config/internal/bb-file-filter-configuration.properties
Change the section:
# Filtered file types
.htm=filter
.html=filter
.css=filter
.js=filter
.txt=filter
.xml=filter
.svg=filter
to
# Filtered file types
.htm=filter
.html=filter
.xhtml=filter
.xhm=filter
.css=filter
.js=filter
.txt=filter
.xml=filter
.svg=filter
.svgz=filter
.xsl=filter

Test change to bb-file-filter-configuration.properties

All commands are run as bbuser.
This is performed on every node
Test 1: File contents
Display file contents
cat /usr/local/blackboard/config/internal/bb-file-filter-configuration.properties;
Confirm the below text appears:
# Filtered file types
.htm=filter
.html=filter
.xhtml=filter
.xhm=filter
.css=filter
.js=filter
.txt=filter
.xml=filter
.svg=filter
.svgz=filter
.xsl=filter
Test 2: Size and modified date
Run the following command on every node.
ls -l /usr/local/blackboard/config/internal/bb-file-filter-configuration.properties;
Confirm:
  1. This file is the same size and date across all nodes

DEV/PREPROD/LIVE Randomness fix

Edit the random source file attribute of the $JAVA_HOME/jre/lib/security/java.security file on each node.
# with staff account, NOT bbuser
# on each node
# confirm it's not bbuser
whoami
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-oracle.x86_64/ ;
adm sed -i 's/securerandom.source=file:\/dev\/random/securerandom.source=file:\/dev\/urandom/' $JAVA_HOME/jre/lib/security/java.security;

DEV/PREPROD/LIVE File ownership

Blackboard files should be owned by bbuser, otherwise upgrades will fail.
#On each node
find /data/bb/blackboard/ -maxdepth 2 -mindepth 2 -type d -not -name content | xargs -I {} chown -R bbuser\: {}

DEV/PREPROD/LIVE Ulimit

The upgrade produced errors due to ulimit being too low. Therefore we should raise the ulimit as follows:
#On each node
#Login as root
- vim /etc/security/limits.conf
- add following lines:
* soft nofile 5000
* hard nofile 65536
#Pay attention that
# does not appear in front of either line - save - logout - login
#as root check: - ulimit -n - ulimit -Hn
[root@~]# ulimit -n 5000 [root@~]
# ulimit -Hn 65536
#switch to bbuser
and check is the same result:
- su bbuser -
ulimit -n -
ulimit -Hn
[bbuser@]$ ulimit -n 5000
[bbuser@]$ ulimit -Hn 65536
Verification
[bbuser@]$ ulimit -n 5000 [bbuser@]$ ulimit -Hn 65536

PREPROD/LIVE Speed up Blackboard start stop

Blackboard files affected:
/usr/local/blackboard/config/tomcat/conf/wrapper.conf
/usr/local/blackboard/config/tomcat/conf/wrapper.conf.bb
  1. In order to reduce time that it takes for Blackboard to services to stop as recommended at https://blackboard.secure.force.com/btbb_articleview?id=kAC7000000000FS, make following change to /usr/local/blackboard/apps/tomcat/conf/wrapper.conf on each node:
#As bbuser
#On each node
sed -i 's/wrapper.shutdown.timeout=300/wrapper.shutdown.timeout=30/' /usr/local/blackboard/config/tomcat/conf/wrapper.conf ;
sed -i 's/wrapper.jvm_exit.timeout=15/wrapper.jvm_exit.timeout=5/' /usr/local/blackboard/config/tomcat/conf/wrapper.conf ;
sed -i 's/wrapper.ping.timeout=60/wrapper.ping.timeout=90/' /usr/local/blackboard/config/tomcat/conf/wrapper.conf ; sed -i 's/wrapper.shutdown.timeout=300/wrapper.shutdown.timeout=30/' /usr/local/blackboard/config/tomcat/conf/wrapper.conf.bb ;
sed -i 's/wrapper.jvm_exit.timeout=15/wrapper.jvm_exit.timeout=5/' /usr/local/blackboard/config/tomcat/conf/wrapper.conf.bb ;
sed -i 's/wrapper.ping.timeout=60/wrapper.ping.timeout=90/' /usr/local/blackboard/config/tomcat/conf/wrapper.conf.bb
Verification
#On each node
cat /usr/local/blackboard/config/tomcat/conf/wrapper.conf |grep wrapper.shutdown.timeout ;
cat /usr/local/blackboard/config/tomcat/conf/wrapper.conf |grep wrapper.jvm_exit.timeout ;
cat /usr/local/blackboard/config/tomcat/conf/wrapper.conf |grep wrapper.ping.timeout
#result should be
wrapper.shutdown.timeout=30
wrapper.jvm_exit.timeout=5
wrapper.ping.timeout=90
#On each node
cat /usr/local/blackboard/config/tomcat/conf/wrapper.conf.bb |grep wrapper.shutdown.timeout ;
cat /usr/local/blackboard/config/tomcat/conf/wrapper.conf.bb |grep wrapper.jvm_exit.timeout ;
cat /usr/local/blackboard/config/tomcat/conf/wrapper.conf.bb |grep wrapper.ping.timeout
#result should be wrapper.shutdown.timeout=30 wrapper.jvm_exit.timeout=5 wrapper.ping.timeout=90

PREPROD/LIVE - Xythos Cross Server Communication Fix

This issue is fixed in 2017 Q4 and should no longer need to be applied.
Blackboard files affected:
/usr/local/blackboard/config/tomcat/conf/server.xml.bb
  1. Apply xythos cross server communication fix on each node.
    1. On each node
    2. Backup file
    cp /usr/local/blackboard/config/tomcat/conf/server.xml.bb /home/bbuser/server.xml.bb.$(date +"%Y-%m-%d")
    1. On each node
    2. Edit /usr/local/blackboard/config/tomcat/conf/server.xml.bb
    nano /usr/local/blackboard/config/tomcat/conf/server.xml.bb
    1. find the stanza (near end of file around lines 203 - 207) defining the xythos admin connector and add secure="true" i.e.
    2. Currently it is:
    <Connector port="@bbconfig.xythosremoteadmin.xythos.servergroup.portnumber@"
    protocol="org.apache.coyote.http11.Http11NioProtocol"
    executor="crossServerExecutor"
    backlog="@@bbconfig.xythosremoteadmin.connector.backlog@@"
    enableLookups="false"
    keepAliveTimeout="@@bbconfig.appserver.http.keepalive.timeout@@"
    maxKeepAliveRequests="@@bbconfig.appserver.http.keepalive.requests@@"
    maxPostSize="@@bbconfig.appserver.max.post.size@@"
    maxParameterCount="@@bbconfig.appserver.max.parameter.count@@"
    soTimeout="@@bbconfig.appserver.http.so.timeout@@"
    URIEncoding="UTF-8"/>
    1. Change to:
    <Connector port="@bbconfig.xythosremoteadmin.xythos.servergroup.portnumber@"
    protocol="org.apache.coyote.http11.Http11NioProtocol"
    executor="crossServerExecutor"
    backlog="@@bbconfig.xythosremoteadmin.connector.backlog@@"
    enableLookups="false"
    keepAliveTimeout="@@bbconfig.appserver.http.keepalive.timeout@@"
    maxKeepAliveRequests="@@bbconfig.appserver.http.keepalive.requests@@"
    maxPostSize="@@bbconfig.appserver.max.post.size@@"
    maxParameterCount="@@bbconfig.appserver.max.parameter.count@@"
    soTimeout="@@bbconfig.appserver.http.so.timeout@@"
    URIEncoding="UTF-8"
    secure="true"/> <----- Part to add
  2. New method using sed
    # as bbuser
    # on each node

    sed -i 's;URIEncoding="UTF-8"/>;URIEncoding="UTF-8" \n secure="true"/>;g' /usr/local/blackboard/config/tomcat/conf/server.xml.bb
  3. Verification step of above on each node.
    #on each node
    cat /usr/local/blackboard/config/tomcat/conf/server.xml.bb |grep secure
    secure="true"
    secure="true"
    secure="true"/>

DEV/PREPROD/LIVE Resolve Math Editor not loading issue

Blackboard files affected:
/usr/local/blackboard/config/internal/http-cacheable-extensions.txt

Issue described at https://blackboard.secure.force.com/btbb_articleview?id=kA370000000H4uu can be resolved by:
  1. On each node update the http-cacheable-extensions.txt to include reference to resolve issues with Equation editor.
#on each node
#as bbuser
cd /usr/local/blackboard/config/internal/ nano http-cacheable-extensions.txt
#If not already present, add the following to the list in their alphabetical place: eot ttf woff
#and save it.
Verification
#As bbuser
#On each node
grep 'eot\|ttf\|woff' /usr/local/blackboard/config/internal/http-cacheable-extensions.txt | wc -l
#Result must equal 3.

PREPROD/LIVE Workaround to issue LRN-107071: High CPU and High Load Caused by Script MicrosoftDocumentParser.sh

Blackboard files affected:
/usr/local/blackboard/config/internal/xythos-indexing-filter.txt
/usr/local/blackboard/config/tomcat/classes/Search.properties
/usr/local/blackboard/config/tomcat/classes/Search.properties.bb

LRN-107071: High CPU and High Load Caused by Script MicrosoftDocumentParser.sh is an issue that should be resolved in 2017 Q2, so this workaround should not be needed in future upgrades.
  1. Stop Blackboard on all nodes, web first, then tasks node
  2. Apply database fix to CMS schema
    UPDATE XY_SERVER_GROUP_PARAMETERS
    SET PARAMETER_VALUE = 'exe,dll,zip,jpg,gif,tif,tiff,dmp,pdf,doc,docx,ppt,pps,pptx,ppsx,ppts,xlsx,xls'
    WHERE PARAMETER_NAME = 'Xythos.Search.ExtensionsNotToIndex';

    UPDATE XY_SERVER_GROUP_PARAMETERS
    SET PARAMETER_VALUE = 'application/pdf,application/x-pdf,application/vnd.ms-excel,
    application/msexcel,
    application/x-msexcel,application/x-ms-excel,
    application/x-excel,
    application/x-dos_ms_excel
    ,application/xls,
    application/x-xls
    ,application/doc,
    application/x-doc,
    application/msword
    ,application/ppt,
    application/x-ppt,
    application/vnd.ms-powerpoint,
    application/pps
    ,application/x-pps
    ,application/vnd.openxmlformats-officedocument.spreadsheetml.sheet,
    application/vnd.openxmlformats-officedocument.wordprocessingml.document
    ,application/vnd.openxmlformats-officedocument.presentationml.presentation,
    application/vnd.openxmlformats-officedocument.presentationml.slideshow'
    WHERE PARAMETER_NAME = 'Xythos.Search.MIMETypesNotToIndex';
  3. Apply workaround to issue LRN-107071: High CPU and High Load Caused by Script MicrosoftDocumentParser.sh by adding lines to /usr/local/blackboard/config/internal/xythos-indexing-filter.txt (note that Blackboard claim this issue is resolved in 9.1 2017 Q2)
    1. On each node
    2. Backup file
    cp /usr/local/blackboard/config/internal/xythos-indexing-filter.txt /home/bbuser/xythos-indexing-filter.txt.$(date +"%Y-%m-%d")
    1. On each node
    2. Edit /usr/local/blackboard/config/internal/xythos-indexing-filter.txt
    nano /usr/local/blackboard/config/internal/xythos-indexing-filter.txt
    1. Set the whole file to contain
    # This file contains mime types that should be filtered from Xythos indexing
    # This file contains mime types that should be filtered from Xythos indexing
    application/doc
    application/msword
    application/msexcel
    application/pps
    application/ppt
    application/vnd.ms-excel
    application/vnd.ms-powerpoint
    application/vnd.openxmlformats-officedocument.presentationml.presentation
    application/vnd.openxmlformats-officedocument.presentationml.slideshow
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
    application/vnd.openxmlformats-officedocument.wordprocessingml.document
    application/x-doc
    application/x-dos_ms_excel
    application/x-excel
    application/xls
    application/x-msexcel
    application/x-ms-excel
    application/x-pps
    application/x-ppt
    application/x-xls
Verification
#as bbuser
cat /usr/local/blackboard/config/internal/xythos-indexing-filter.txt
#output that should be returned:
# This file contains mime types that should be filtered from Xythos indexing
application/doc
application/msword
application/msexcel
application/pps
application/ppt
application/vnd.ms-excel
application/vnd.ms-powerpoint
application/vnd.openxmlformats-officedocument.presentationml.presentation
application/vnd.openxmlformats-officedocument.presentationml.slideshow
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
application/vnd.openxmlformats-officedocument.wordprocessingml.document
application/x-doc application/x-dos_ms_excel
application/x-excel
application/xls
application/x-msexcel
application/x-ms-excel
application/x-pps
application/x-ppt
application/x-xls
  1. Adjust Search.properties files accordingly on each node.
#as bbuser
#on each node
sed -i 's/ExtensionsNotToIndex=exe,dll,zip,jpg,gif,tif,tiff,dmp/ExtensionsNotToIndex=exe,dll,zip,jpg,gif,tif,tiff,dmp,pdf,doc,docx,ppt,pps,pptx,ppsx,ppts,xlsx,xls,xlsm/' /usr/local/blackboard/config/tomcat/classes/Search.properties ;
sed -i 's/ExtensionsNotToIndex=exe,dll,zip,jpg,gif,tif,tiff,dmp/ExtensionsNotToIndex=exe,dll,zip,jpg,gif,tif,tiff,dmp,pdf,doc,docx,ppt,pps,pptx,ppsx,ppts,xlsx,xls,xlsm/' /usr/local/blackboard/config/tomcat/classes/Search.properties.bb

PREPROD/LIVE Adjust logging and task management

Blackboard files affected:
/usr/local/blackboard/config/service-config.properties

Our Blackboard environment has four web nodes and a tasks node. We will set the tasks node to run tasks and the web nodes to not run tasks. We will set the logging on the web nodes to show fatal errors only, whilst the tasks node will log warnings as well (default setting).
N.B. To set a node to run tasks it is also necessary to set bbconfig.frontend.fullhostname=true in the bb-config.properties file on that node. This setting is honoured by the installer, but the following additional necessary setting is wiped by the installer.
Edit the /usr/local/blackboard/config/service-config.properties on all nodes EXCEPT tasks node to ensure that tasks node is the only node set to carry out background tasks, thus optimising performance of web facing end user nodes.
#As bbuser on all nodes except tasks node
#Edit the file /usr/local/blackboard/config/service-config.properties
nano /usr/local/blackboard/config/service-config.properties
#and alter the line
#blackboard.service.queuedtaskmanager.param.num-concurrent-tasks=2
#to read
#blackboard.service.queuedtaskmanager.param.num-concurrent-tasks=0
# using sed:
# on web nodes only:
sed -i 's/blackboard.service.queuedtaskmanager.param.num-concurrent-tasks=2/blackboard.service.queuedtaskmanager.param.num-concurrent-tasks=0/' /usr/local/blackboard/config/service-config.properties
#and change logging level to fatal on all nodes but tasks node
blackboard.service.log.param.logdef.services.verbosity=fatal
# using sed:
# on web nodes only:
sed -i 's/blackboard.service.log.param.logdef.services.verbosity=warning/blackboard.service.log.param.logdef.services.verbosity=fatal/' /usr/local/blackboard/config/service-config.properties
Verification
On web nodes
cat /usr/local/blackboard/config/service-config.properties |grep blackboard.service.queuedtaskmanager.param.num-concurrent-tasks ;
cat /usr/local/blackboard/config/service-config.properties |grep blackboard.service.log.param.logdef.services.verbosity
#should return
blackboard.service.queuedtaskmanager.param.num-concurrent-tasks=0
blackboard.service.log.param.logdef.services.verbosity=fatal
On tasks node
cat /usr/local/blackboard/config/service-config.properties |grep blackboard.service.queuedtaskmanager.param.num-concurrent-tasks ;
cat /usr/local/blackboard/config/service-config.properties |grep blackboard.service.log.param.logdef.services.verbosity
#should return
blackboard.service.queuedtaskmanager.param.num-concurrent-tasks=2
blackboard.service.log.param.logdef.services.verbosity=warning

DEV/PREPROD/LIVE Raise Notification Distribution setting

Blackboard files affected:
/usr/local/blackboard/config/internal/nautilus-config.properties

Raise the Notification Distribution setting to ensure more timely notification delivery.
On each node, edit /usr/local/blackboard/config/internal/nautilus-config.properties and change nautilus.distribution.notificationsPerDistribution=10000 to 20000
#as bbuser
#on each node
sed -i 's/nautilus.distribution.notificationsPerDistribution=10000/nautilus.distribution.notificationsPerDistribution=20000/' /usr/local/blackboard/config/internal/nautilus-config.properties
Verification
cat /usr/local/blackboard/config/internal/nautilus-config.properties | grep "nautilus.distribution.notificationsPerDistribution=20000" | wc -l
#answer should be 1

DEV/PREPROD/LIVE Improve Tool Settings Changes Time To Take Effect

This resolves an issue where changes to Tool Settings can take a long time do take effect across all nodes, see https://blackboard.secure.force.com/btbb_articleview?id=kA370000000H0jw. Blackboard files affected:
/usr/local/blackboard/config/cache-settings.properties

Reduce time for tool changes to take effect.


On each node, edit /usr/local/blackboard/config/cache-settings.properties and reduce bbconfig.cache.contextualizedNodeAffiliates.timetoidle=345600 to 1800 and reduce bbconfig.cache.default.timetoidle=172800 to 21600
#as bbuser
#on each node
sed -i 's/bbconfig.cache.contextualizedNodeAffiliates.timetoidle=345600/bbconfig.cache.contextualizedNodeAffiliates.timetoidle=1800/' /usr/local/blackboard/config/cache-settings.properties ;
sed -i 's/bbconfig.cache.default.timetoidle=172800/bbconfig.cache.default.timetoidle=21600/' /usr/local/blackboard/config/cache-settings.properties
Verification
cat /usr/local/blackboard/config/cache-settings.properties | grep "bbconfig.cache.contextualizedNodeAffiliates.timetoidle=1800" | wc -l #answer should be 1 cat /usr/local/blackboard/config/cache-settings.properties | grep "bbconfig.cache.default.timetoidle=21600" | wc -l
#answer should be 1

DEV/PREPROD/LIVE Resolve "ActiveMQ Memory Limit Causing SIS Failures" Known Issue

The ActiveMQ message queue can grow past the amount of memory allocated to it. This causes a back up of messages, and will prevent SIS integrations from being processed. This can be resolved by raising a couple of values in the message queue configuration files. See https://blackboard.secure.force.com/btbb_articleview?id=kA339000000CfVn

Blackboard files affected:
/usr/local/blackboard/config/message-queue-service-config.xml
/usr/local/blackboard/config/message-queue-service-config.xml.bb

On each node, edit /usr/local/blackboard/config/message-queue-service-config.xml and /usr/local/blackboard/config/message-queue-service-config.xml.bb and increase memoryUsage limit from 256 MB to 512 MB and increase storeUsage limit from 1g to 2g
#as bbuser
#on each node
sed -i 's/memoryUsage limit="256 MB"/memoryUsage limit="512 MB"/' /usr/local/blackboard/config/message-queue-service-config.xml ;
sed -i 's/storeUsage limit="1g"/storeUsage limit="2g"/' /usr/local/blackboard/config/message-queue-service-config.xml ;
sed -i 's/memoryUsage limit="256 MB"/memoryUsage limit="512 MB"/' /usr/local/blackboard/config/message-queue-service-config.xml.bb ;
sed -i 's/storeUsage limit="1g"/storeUsage limit="2g"/' /usr/local/blackboard/config/message-queue-service-config.xml.bb
Verification
cat /usr/local/blackboard/config/message-queue-service-config.xml | grep "memoryUsage limit=\"512 MB\"" | wc -l
#answer should be 1
cat /usr/local/blackboard/config/message-queue-service-config.xml | grep "storeUsage limit=\"2g\"" | wc -l
#answer should be 1
cat /usr/local/blackboard/config/message-queue-service-config.xml.bb | grep "memoryUsage limit=\"512 MB\"" | wc -l
#answer should be 1
cat /usr/local/blackboard/config/message-queue-service-config.xml.bb | grep "storeUsage limit=\"2g\"" | wc -l
#answer should be 1

DEV/PREPROD/LIVE Forum is enabled issue

According to https://blackboard.secure.force.com/btbb_articleview?id=kA3390000008aEg usage of the discussion board can caused high DB load. Whilst we await a fix Blackboard have provided a replacement file that resolves the issue.

Blackboard files affected:
/usr/local/blackboard/system/database/vi/discussionboard/functions/forum_is_enabled.sql

Current File location: https://mle.southampton.ac.uk/bb/bb2018-fixes/forum_is_enabled.sql

Process:
  1. MLE: Get the updated "forum_is_enabled.sql" file (see https://blackboard.secure.force.com/btbb_caseview?id=5003900001zkPvwAAE) and place in bbuser's home folder on each node.
    # on each node
    # as bbuser

    cd ;
    wget https://mle.southampton.ac.uk/bb/bb2018-fixes/forum_is_enabled.sql
  2. APPSMAN: After upgrading Blackboard, back up the forum_is_enabled.sql file.
    cp /usr/local/blackboard/system/database/vi/discussionboard/functions/forum_is_enabled.sql /home/bbuser/post_upgrade_files/
  3. APPSMAN: On each node copy the new forum_is_enabled.sql file.
    # on each node
    # as bbuser

    cp /home/bbuser/forum_is_enabled.sql /usr/local/blackboard/system/database/vi/discussionboard/functions/forum_is_enabled.sql

DEV/PREPROD/LIVE Safari upload issue

This is not needed from 2018 Q2 onwards
According to https://blackboard.secure.force.com/btbb_articleview?id=kA3390000008bOi users of Safari 11.1 are unable to upload files to Blackboard. Blackboard have provided a fixed widget.js file that resolves the problem.

Blackboard files affected:
/usr/local/blackboard/docs/javascript/ngui/widget.js

Current File location: https://mle.southampton.ac.uk/bb/bb2018-fixes/widget.js

Process:
  1. MLE: Get the updated "widget.js" file (see https://blackboard.secure.force.com/btbb_articleview?id=kA3390000008bOi) and place in bbuser's home folder on each node.
    # on each node
    # as bbuser

    cd ;
    wget https://mle.southampton.ac.uk/bb/bb2018-fixes/widget.js
  2. APPSMAN: After upgrading Blackboard, back up the forum_is_enabled.sql file.
    cp /usr/local/blackboard/docs/javascript/ngui/widget.js /home/bbuser/post_upgrade_files/
  3. APPSMAN: On each node copy the new forum_is_enabled.sql file.
    # on each node
    # as bbuser

    cp /home/bbuser/widget.js /usr/local/blackboard/docs/javascript/ngui/widget.js

List of affected files

Files where changes are made in DEV environments

/usr/local/blackboard/config/bb-config.properties
/usr/local/blackboard/config/bb-config.properties.template
/usr/local/blackboard/config/internal/http-cacheable-extensions.txt
/usr/local/blackboard/config/internal/nautilus-config.properties
/usr/local/blackboard/config/cache-settings.properties
/usr/local/blackboard/config/message-queue-service-config.xml
/usr/local/blackboard/config/message-queue-service-config.xml.bb
/usr/local/blackboard/content/vi/BBLEARN1/plugins/bb-xss-filter/webapp/WEB-INF/classes/blackboard/xss/regex/bb-xss-regex-filter.properties
/usr/local/blackboard/config/internal/bb-file-filter-configuration.properties
/usr/local/blackboard/system/database/vi/discussionboard/functions/forum_is_enabled.sql
/usr/local/blackboard/docs/javascript/ngui/widget.js

Files where changes are made in PREPROD/LIVE environments

/usr/local/blackboard/config/bb-config.properties
/usr/local/blackboard/config/bb-config.properties.template
/usr/local/blackboard/config/tomcat/conf/wrapper.conf
/usr/local/blackboard/config/tomcat/conf/wrapper.conf.bb
/usr/local/blackboard/config/tomcat/conf/server.xml.bb
/usr/local/blackboard/config/internal/http-cacheable-extensions.txt
/usr/local/blackboard/config/internal/xythos-indexing-filter.txt
/usr/local/blackboard/config/tomcat/classes/Search.properties
/usr/local/blackboard/config/tomcat/classes/Search.properties.bb
/usr/local/blackboard/config/service-config.properties
/usr/local/blackboard/config/internal/nautilus-config.properties
/usr/local/blackboard/config/cache-settings.properties
/usr/local/blackboard/config/message-queue-service-config.xml
/usr/local/blackboard/config/message-queue-service-config.xml.bb
/usr/local/blackboard/content/vi/bb_bb60/plugins/bb-xss-filter/webapp/WEB-INF/classes/blackboard/xss/regex/bb-xss-regex-filter.properties
/usr/local/blackboard/config/internal/bb-file-filter-configuration.properties
/usr/local/blackboard/system/database/vi/discussionboard/functions/forum_is_enabled.sql
/usr/local/blackboard/docs/javascript/ngui/widget.js

Comments

Post a Comment

Popular posts from this blog

Should you use Significant Figures or Decimal Places when setting Answer Set options for Calculated Formula questions in Blackboard?

-
Image
Blackboard tests are a hot topic at the moment. I recently recently received a question from Dr Gabriel Weymouth : "I'm positive this is clearly documented somewhere, but I can't find it. How do you make BB apply the correct error margins for calculated formula problems. The logical way of doing it (setting a, say, 2% margin) doesn't work:"
Read more