Many older web applications do not apply headers/tags that are now considered standard information security practices. For example:
Pragma: no-cache
Cache-Control: no-cache
httpOnly and secure flags
Adding these controls can be achieved using ModSecurity without any needs to modify the application code.
In the case where I needed to modify the cookie headers to include these now controls I added the following to core rule set file: modsecurity_crs_16_session_hijacking.conf.
#
# This rule will identify the outbound Set-Cookie SessionID data and capture it in a setsid
#
#addding httpOnly
Header edit Set-Cookie "(?i)^(JSESSIONID=(?:(?!httponly).)+)$" "$1; httpOnly"
Header set Cache-Control "no-cache, no-store, must-revalidate"
Header set Pragma "no-cache"
Header set Expires "0"
This adds the cookie controls we were after – Depending on your web application you may need to change ‘JSESSIONID’ to the name of the relevant cookie.
You can find the cookie name simply using browser tools such as Chrome’s Developer Tools (hit F12 in chrome). Load the page you want to check cookies for, click on the Resources tab:
After setting the HTTPOnly and Secure flags you can check the effectiveness using the Console table and listing the document cookies… which should now return nothing.
Distinguished name order of openssl may be opposite of ejbca default configuration – http://www.csita.unige.it/software/free/ejbca/ … If so, this ordering must changed in ejbca configuration prior to deploying (can’t be set on a per CA basis)
Have not been able to replicate this issue in testing.
Import existing TinyCA CA
Basic Admin and User operations
Create and end entity profile for server/client entities
Step 2 – Sign CSR using the End Entity which is associated with a CA
Importing existing certificates
EJBCA can create endentities and import their existing certificate one-by-one or in bulk (http://www.ejbca.org/docs/adminguide.html#Importing Certificates). Bulk inserts import all certificates under a single user which may not be desirable. Below is a script to import all certs in a directory one by one under a new endentity which will take the name of the certificate CN.
#!/bin/sh
# for each certificate in the directory
# create and enduserentity
# enduserentity username = certificate CN
# enduserentity token/pwrd = certificate CN
EJBCA_HOME="/usr/share/ejbca"
IMPORT_DIR=$1
CA=$2
ENDENTITYPROFILE=$3
SSLCERTPROFILE=$4
AP="_OTE"
if [ $# -lt 4 ]; then
echo "usage: import_existing_certs.sh "
exit 1
fi
for X in $IMPORT_DIR*.pem
do
echo "######################################################"
echo "Importing: " $X
CN=$(openssl x509 -in $X -noout -text | grep Subject: | sed -n 's/^.*CN=\(.*\),*/\1/p')
echo "CN: " $CN
printf "Running import: %s ca importcert '%s' '%s' '%s' ACTIVE NULL '%s' '%s' '%s'\n" "$EJBCA_HOME/bin/ejbca.sh" "$CN" "$CN" "$CA" "$X" "$ENDENTITYPROFILE" "$SSLCERTPROFILE"
$EJBCA_HOME/bin/ejbca.sh ca importcert "$CN$AP" "$CN$AP" "$CA" ACTIVE null $X $ENDENTITYPROFILE $SSLCERTPROFILE
echo "######################################################"
done
#Generate CRL via command line
# List CAs
/usr/share/ejbca/bin/ejbca.sh CA listcas
# Create new CRLs:
/usr/share/ejbca/bin/ejbca.sh CA createcrl "" -pem
# Export CRL to file
/usr/share/ejbca/bin/ejbca.sh CA getcrl "" -pem .pem
Checking certificate validity/revoke status via OSCP
XSS, CSRF and similar types of web application attacks have overtaken SQL injections as the most commonly seen attacks on the internet (https://info.cenzic.com/2013-Application-Security-Trends-Report.html). A very large number of web application were written and deployed prior to the trend up in likelihood and awareness of XSS attacks. Thus, it is extremely important to have an effective method of testing for XSS vulnerabilities and mitigating them.
Changes to production code bases can be slow, costly and can miss unreported vulnerabilities quite easily. The use of application firewalls such as ModSecurity (https://github.com/SpiderLabs/ModSecurity/) become an increasingly attractive solution when faced with a decision on how to mitigate current and future XSS vulnerabilities.
Mod Security can be embedded with Apache, NGINX and IIS which is relativity straight forward. In cases where alternative web severs are being used ModSecurity can still be a viable option by creating a reverse proxy (using Apache of NGINX).
A default action can be created for a group of rules using the configuration directive “SecDefaultAction”
Using the following SecDefaultAction at the top of rule set that we want enable blocking and transforming on is a blunt method of protection. Redirection can also be used as a method of blocking.
A powerful web application firewall – free software!
Example of a default action to be applied by ruleset (note defaults cascade through the ruleset files):
Using optional rulesets, modsecurity_crs_16_session_hijacking.conf and modsecurity_crs_43_csrf_protection.conf ModSecurity can provide protection against Cross Site Request Forgeries [CSRF]. The @rsub operators can inject a token on every form (and/or other html elements). ModSecurity can store the expected token value as a variable which is compared to the value posted via forms or other html elements. ModSecurity rules can be based on request methods and URIs etc – alongside the ability to chain rules there are a huge number of options for mitigating XSS and CSRF without impacting normal applicatioin usage.
@rsub
Requirements:
SecRuleEngine On
SecRequestBodyAccess On
SecResponseBodyAccess On
## To enable @rsub
SecStreamOutBodyInspection On
SecStreamInBodyInspection On
SecContentInjection On
Injecting unique request id from mod_unique_id into forms:
<LocationMatch ".*\/<directory requiring authentication>\/.*">
# All requests submitted using POST require a token - not the validation of the token can only be completed if that variable is stored from a previous response
SecRule REQUEST_METHOD "^(?:POST)$" "chain,phase:2,id:'1234',t:none,block,msg:'CSRF Attack Detected - Missing CSRF Token when using POST method - ',redirect:/" SecRule &ARGS:token "!@eq 1" "setvar:'tx.msg=%{rule.msg}',setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},setvar:tx.%{rule.id}-WEB_ATTACK/CSRF-%{matched_var_name}=%{matched_var}"
# Check referrer is valid for an authenticated area of the application
SecRule REQUEST_HEADERS:Referer "!@contains <my website>" "block,phase:2,id:'2345',t:none,block,msg:'CSRF Attack Detected - No external referers allowed to internal portal pages',redirect:/"
SecRule REQUEST_URI "@contains confirmUpdate" "chain,phase:2,id:'3456',t:none,block,msg:'CSRF Attack Detected - Missing CSRF Token. Confirmation button - ',redirect:/" SecRule &ARGS:rv_token "!@eq 1" "setvar:'tx.msg=%{rule.msg}',setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},setvar:tx.%{rule.id}-WEB_ATTACK/CSRF-%{matched_var_name}=%{matched_var}"
</LocationMatch>
Pros:
Wide capabilities for logging, alerts, blocking, redirecting, transforming
Parses everything coming into your web server over HTTP
Virtual patching – if a vulnerability is made public that affects your web application you can write and deploy a rule to mitigate the vulnerability much faster than re-release of application code patched
Extended uses – the capabilities of ModSecurity can be applied to applications outside the scope of application security
Cons:
Added complexity to your application delivery chain – another point for maintenance and failure
Performance costs? – Though I have not had the opportunity to test the performance costs holding session information in memory and inspecting every byte of HTTP traffic can’t be free from performance cost
Hardware costs – Particularly if using ModSecurity’s BodyAccess and BodyInspection features, memory usage will be significant
Improving deployments:
Starting off being aggressive on warnings and very light on action is a necessity to ensure no impact on normal application usage
From this point rules and actions need to be refined
Understanding how the applications works allows the use of ModSecuirtys header and body inspection in effective ways
Some other notes extracted from the ModSecurity Handbook – If you decide to use ModSecurity I strongly recommend buying the handbook. It is not expensive and saves a lot of time.
### STRING MATCHING OPERATORS ###
@beginsWith Input begins with parameter
@contains Input contains parameter
@endsWith Input ends with parameter
@rsub Manipulation of request and response bodies
@rx Regular pattern match in input
@pm Parallel pattern matching
@pmFromFile (also @pmf as of 2.6) Parallel patterns matching, with patterns read from a file
@streq Input equal to parameter
@within Parameter contains input
### NUMBER MATCHING OPERATORS ###
@eq Equal
@ge Greater or equal
@gt Greater than
@le Less or equal
@lt Less than
### ACTIONS ###
# DISRUPTIVE
allow Stop processing of one or more remaining phases
block Indicate that a rule wants to block
deny Block transaction with an error page
drop Close network connection
pass Do not block, go to the next rule
pause Pause for a period of time, then execute allow.
proxy Proxy request to a backend web server
redirect Redirect request to some other web server
# FLOW
chain Connect two or more rules into a single logical rule
skip Skip over one or more rules that follow
skipAfter Skip after the rule or marker with the provided ID
Most of us use and rely on SSL everyday. The mathematical workings of the RSA [Rivest, Shamir, Adleman] algorithm are not overly complex but mapping everything back to what happens in reality requires detailed understanding. Skipping over the need for SSL (for confidential and authenticated exchange of a symmetric key over and insecure medium) I will review the mathematical workings then how they are applied in real world examples.
There are also details in previous posts – RSA1, RSA2
RSA decrypt -> ciph ^ d mod n = 3095021178047041558314072884014000324030086129008597834642883051983162360819331 ^ 944402082567056818708092537028397604145319798848072425038015030084640082599681 mod
When Alice encrypts using Bob’s public key (e) along with the key modulus (n) the output is a protected cipher.
An eavesdropper does not know the private key so decryption is very difficult:
Attacker must solve:
(unknown val, x) ^ e mod n = ciph
x ^65537 mod 4052729130775091849638047446256554071699019514021047339267026030072286291982163 = 3095021178047041558314072884014000324030086129008597834642883051983162360819331
OR, easier – try to determine the private key:
The attacker knows e and n (which = pq). When we created the private key (step 6 above) we conducted: e–1 mod φ(n) – Modular multiplicative inverse which is relatively fast for us to calculate.
The attacked does not know e–1 mod φ(n) though. φ(n) = (p – 1)(q – 1). The attacker knows that n is a composite prime = pq (where p and q are both primes).
So… if the attacker can solve p * q = n (where they know n) then RSA is insecure.
Thankfully the process of Integer factorization is so much harder than the process of creating p,q,n, φ(n), e and d thatonline business and confidentiality can be maintained to acceptable levels.
Threats to RSA
It would be extremely valuable to malicious individuals/groups and (more importantly) intelligence organizations make large integer factorization efficient enough to break RSA.
Usage: sh GetPlaylist.sh [youtube playlist URL] [output_directory]
Example:
sh ./GetYoutubePlaylist.sh http://www.youtube.com/playlist?list=PL702CAF4AD2AED35B ~/Music
To get the youtube playlist URL view the playlist by clicking on its name, no videos will be playing, then click on share and the url will be highlighted:
Step 4: Write an interface between the functions and a basic interface (text?)
Step 5: Test!
Conclusion: Using Scala (trying anyway) the implementation is not very complex. What is more difficult is replicating the original machines where the output character could not equal the input character. The limitation Turing used to crack the machines. Might see if I can implement that later then test out the cracking method. Not sure how easy it would be to break the current implementation…
After hearing the praises of Hadoop I had a brief look it and the Map/Reduce paradigm. Most info was read from Wikipedia and references in wiki articles. In particular, this paper from Jeffrey Dean and Sanjay Ghemawat.
Hadoop is an opensource software framework that implements MapReduce and Google File System. The aim is to enable easy and robust deployment of highly parallel data set processing programs. It is important to note that the MapReduce model is applicable to embarrassingly parallel problems. Processing can occur on data that is stored in a database or filesystem.
Map/Reduce refers to the steps in the model:
Map: A master node takes an input problem and divides it into sub-problems, passing them to worker nodes. Worker nodes can further divide sub-problems. For example, in the problem of counting word occurrence in a document the Map function will output a key/value pair every time it sees a a specified work – ie: (“searchterm”, 1).
Reduce: The reduce function takes the list of word(key)/values and sums the occurrences:
(foo, (1)
(bar, 1)
(foo, 1)
(bar, 1)
(foo, 1)
The output of reduce in this case could be: (foo, 3).
The MapReduce model becomes powerful when considering giant datasets can be processed in large clusters very efficiently using this model.
Hadoop Run-time Takes care of:
Parallelalization
Partitioning of input data
Scheduling programs execution across machines
Handling machine failures
Managing inter-machine communication
Hadoop aims to enable developers with little distributed programming experience to utilize compute resources such as EC2.
With the emergence of ‘big data’ and the apparent value that can be extrapolated from massive databases/datastores, many organisations have found the limits of traditional relational databases. Hadoop has such a big buzz because it can pass the processing boundaries of relational database software and enable the extrapolation of value. The video below is a decent explanation of this point by data scientists at SalesForce.com
Came across an interesting scenario this week where I decided to try and use Python for parsing HTML. Turns out to quite straight forward and something I could imagine using more often in the future.
The link we were after was the most viewed channel at any particular time:
The most viewed channel is always in this location/element
First attempt was to load up that page in an <iframe> allowing our own JavaScript to grab the relevant URL then forward. Twitch.tv however forces forwarding when iframes are detected, presumably for proprietary reasons. So javascript was not really an option.
After experimenting with Python’s lxml http://lxml.de/ it was really straight forward and effective. It is apparently quite efficient using C core (http://stackoverflow.com/a/6494811/692180) too. The module I used is documented quite well here: http://lxml.de/lxmlhtml.html. With a little bit of trial an error a brief python script successfully grabs the relevant information.
Then using PHP’s popen() method I can simply call the python script and use the return value as a php variable for a header redirect.
import urllib
from lxml import html
def getElement():
f = urllib.urlopen("http://www.twitch.tv/directory/StarCraft%20II:%20Wings%20of%20Liberty")
# Read from the object, storing the page's contents in 's'.
s = f.read()
f.close()# Read from the object, storing the page's contents in 's'.
doc = html.document_fromstring(s)
doc = doc.get_element_by_id('directory_channels')
#rVal = trimDoc.text_content()
doc = doc.find_class('thumb')
rVal = html.tostring(doc[0]).split()[2]
return makeUrl(rVal)
def makeUrl(thumbString):
return "http://twitch.tv" + thumbString[6:-2]
if __name__ == "__main__":
print getElement()
Wrote a quick script that attempts to complete pascals triangle recursively, thanks Feng.
** Note, after getting some feedback made a number of changes, thanks Fry**
see source: pascals.py
######################################################### # Python Script: Pascals triangle # # Enter How many rows you want... # # Mark Culhane 2011, straightIT.com # #########################################################
