Category: Uni notes

archived posts from previous semesters

Categories
Intelligent systems

FIT5047 – Intelligent Systems Week 7

This week’s lecture delved further into probabilistic systems, specifically Bayesian networks. We explored the rationale behind the explosion in probabilistic systems in the past 15 years, namely the computational shortcuts achieved via the markov property and real world usefulness of  refining probabilities as known information increases.

We got much more detail about Bayesian network this week, including the components:

  • Random Variables [nodes, represented as binary, ordered, integral or continuous states]
  • Node links
  • Conditional Probability table [Quantifying the affect of parent on a  node]
  • The network is a direct, acyclic graph

Nice simple example of a Bayesian network

With the network seen above, causalities are indicated by the node links. If a node state changes from unknown to know, the Bayesian network can efficiently update the probabilities for all the other nodes. Note that the efficiency of a network relies on finding marginally independent nodes. If in this example all of the nodes were linked, the network would not be very effective.  The Markov property as defined by the lecture notes:

There are no direct dependencies in the system being modeled which are not explicitly shown via arcs

In essences when re-propagating probabilities after information is introduced the Markov model allows for encapsulation when completing calculations, speeding up the process.

With a network we can conduct belief updating, a form of inference. Essential to this process is identifying conditional independence of nodes, again this is closely associated to the Markov property. I will need to do some reading before going into more detail about that one. The lecture came to a close with an introduction to Netica, followed up in the tut with some practical experimentation.

 

Categories
Adv. programming for DB apps.

FIT5059 – Adv. Prog. for DB Applications Week 7

Continuation of previous lecture focusing of multiple forms vs multiple canvasses. This is a major concern for distributed development, an area I don’t think it is a strong point for Oracle forms. The source code from tutorial 7:

Custom Forms, Multiple Form Approach
Form 1: Student Form--------------------
1. Goto Tools/Layout Editor2. Create 6 text items (and labels)   e.g. text_item5, text_item6, text_item7, text_item8, text_item29, text_item303. Create 2 push buttons   e.g. push_button20, push_button21
4. Create a PRE-FORM trigger in the Triggers of the Form   :system.message_level :=25;   -- :text_item4 := :global.sid;
5. Create an Alert object   a. Name: NOT_NULL_ALERT   b. Message: Not null error found   c. Alert Style: Stop   d. Button 1 Label: OK
6. Create a program unit called DISPLAY_ALERT
PROCEDURE Display_NOTNULL_ALERT IS  alert_button NUMBER;BEGIN  alert_button := SHOW_ALERT('NOT_NULL_ALERT');  if alert_button = ALERT_BUTTON1 then  	message('Insertion Cancelled');  end if;END;
 
Continue reading "FIT5059 – Adv. Prog. for DB Applications Week 7"
Categories
Natural computation for intell. sys.

FIT5167 – Natural Computation Week 7

Week 7 introduced Genetic Algorithms, who’s effectiveness is somewhat disputed. In any case, these algorithms are quite interesting in their balance between a kind of hill climbing (fitness function) and stochastic methods (cross over, mutation).

The lecture gave the natural basis for these algorithms and defined the key components:

  • Chromosome (ie 101101101)
  • Reproduction (ie crossover/roulette/tournament)
  • Mutation
  • Fitness functions

GAvHillCimb
Genetic Algorithms can find good solutions in large search spaces quickly

The second half of the lecture was dedicated to assignment and unit test revision.

Categories
Network security

FIT5044 – Network Security Week 7

Unfortunately I missed week 7’s lecture thus will be making only a review of the lecture notes. The topic for week 7 was the IP layer and IP security. Using the simplistic network layering of App, Trans, Network, Data Link and Physical [All trains now depart promptly]. We see that the IP is a network level protocol. It is the most common network layer,  a list of other network layers can be found here: http://en.wikipedia.org/wiki/Network_Layer#Protocols. Importantly the function of the network layer is end-to-end routing of packets through networks. The security concerns listed in the lecture were:

  • Authentication
  • Message reply
  • Message alteration
  • Message delay and denial

Some of the original vulnerabilities associated with the IP layer (ping of death and smurf  attacks) were outlined. Next some of the key components of the IP layer were introduced:

  • Bridge
  • Router
  • Gateway
  • Firewall

With the basics out of the way, notes on how to increase the security of the network layer commenced. Authentication, Integrity and Confidentiality were identified and IPSec posed as a  partial solution. Authentication and confidentiality can be enhanced with IPsec through the Authentication Header [AH] and Encapsulating Security Payload [ESP].

 

AH.ESP
Authentication header and Encapsulating Security Payloads

The application of IPSec comes in two modes:

  • Transport mode: Peer-to-peer communications, generally used in internal network comms. Data packets are encrypted, headers are not.
  • Tunnel mode: Header and payload are encrypted, most commonly seen in remote-access.

The remainder of the lecture notes focussed on an explanation of the encryption and key exchanges. A decent summary video which helped me:

Categories
Network security

FIT5044 – Network Security Week Addendum

A couple of useful python scripts I made from copying the explanations on wikipedia:

Extended Euclidean algorithm – Used in RSA for determining the public key, once the private is chosen (see wikipage)

Pollard’s rho algorithm – One method for factorizing (see wikipage), try creating a semi prime with 20 digits then one with 60 digits. Time how long it takes for each 😉

Helpful link with some prime numbers for testing

 

Click on the pic if you don't already have python 2.7

 

Categories
Intelligent systems

FIT5047 – Intelligent Systems Week 6

Intelligent systems’ sixth week took a swing into soft computing and probabilistic systems. We had a quick introduction to probability theory which had the usual intuition breaking outcomes. The use of venn diagrams to explain parts of Kolmogorov’s Axioms were particularly useful. The definition of conditional probability did strike me a little of guard however:

 

Conditional Probability
Conditional Probability

Although in review this does seem much clearer. Given the knowledge of B [yellow] what is the probability of A [red].  As per the diagram and axiom, the answer is the intersection of A and B [green].

A revision of elementary probability reminded me that although at first glance it seems a trivial subject probability requires some use of the brain and calculator:

Suppose a “once in a century” flood has probability 0.01 of occuring in a year. How long do we expect to wait for one?

The answer:

(1 – p)^n = .5

log(1 – p)^n = log (.5)

n log(1 – p) = log (.5)

n = log .5 / log(1 – p)

= approx 69 years

Next came some discussion over notation and then, more importantly, and introduction to Bayes’ Theorm. A simple explination of Bayes’ theorm can be seen here:

 

Discussion then continued to some of the silly mistakes in probability theory that litter its past. I’m sure in 20  years many of the financial tools used in present day will appear on the lecture slides in the same category.

Kevin also made the time to add some material about the nature of probability. The suggestion made in Russell and Norvig is that probability is simply used to represent and agents believe state. If this is kept in mind then it is understandable why Bayesian networks have been such a boon over the past 15 years.

Categories
Adv. programming for DB apps.

FIT5059 – Adv. Prog. for DB Applications Week 6

Week 6 saw an introduction to  custom forms. It makes a lot more sense now why we completed the basic introductions to each component in an automated fashion now. All of the elements that we have already put into practice can be added to a window and customized. Using smart triggers in conjunction with stored procedures gives us the tools that we need to make decent database applications.

The properties palette style customization that is used in Oracle forms developer makes it very similar in feel to ASP.NET development.

More advanced error/exception handling was also discussed this week. Due to the event driven nature of Oracle’s forms, generating exceptions is a convenient way to handle the madness users subject programs to. Customizing systems message will also allow the flow of information from the system to the user to be much more decipherable.

The final topic that was introduced in preparation for next week was the juxtaposition between using multiple canvasses or multiple forms for more complex applications. While multiple forms encompasses much stronger encapsulation, message passing needs to be done using global variables D:<

On the other hand using multiple canvasses allows for much easier message passing, distributed development is decidedly more difficult.

Multi form or multi canvas
Multi form or multi canvas

Doing the assignment will be the best way to ensure that the processes for each element have been memorized.

Categories
Natural computation for intell. sys.

FIT5167 – Natural Computation Week 6

Natural computation entered week 6 with an introduction to unsupervised learning. That is, learning in a neural network without a target for output. This is generally achieved through classification/clustering/self organising maps [SOM].

self organising map
self organising map

The networks for SOMs are actually a little bit simpler than MLP. The process for creating clusters is also quite intuitive. Each neuron in the feature map layer has a unique weight vector, if an input results in that neuron being the most activated (which neuron has the lowest euclidean distance from the input vector) then its weight values move closer to that of the input ( again using euclidian distance):

 

SOM weight update (source: week 6 notes)

The concept of decaying the learning rate was introduced during the lecture but this must be done carefully. If one were to train a network until the weight adjustments stabilized, training will end after a certain number of epochs regardless of how well the network has clustered the data.

Finally the concepts of ‘topological neighborhood’ was introduced. In actual brains, weights of neighboring neurons are updated when a neuron wins the competitive activation. Logically this will result in similar classifications being held by neighboring neurons. The update of the neighboring weight can be done using Gaussian or exponential decay functions:

 

Update neighboring neurons too!
Update neighboring neurons too! (source week 6 notes)
Categories
Network security

FIT5044 – Network Security Week 6

Week 6 continued on the topic of firewalls. We got into some more detailed discussion about the implementation of the concept of firewalls. For example, the use of hardware/software and integration into the OS kernel etc. The question of where and how the firewall engine should be implemented generates a number of options. One thing to remember is that (all else being equal) a more secure configuration will result in less convenience for the business/users. The requirements in terms of security and convenience of each case must be the driving force behind decisions such as the FE implementation.

Next weeks lecture will be focussed on the IP layer and IP security. This week the focus for network security has been the RSA assignment so this is a short post!

TCP/IP stack
Categories
Intelligent systems

FIT5047 – Intelligent Systems Week 5

Taking a turn away from first order logic, search algorithms and planning, week 5 introduced the key issues around natural language processing [NLP] and the programming language Prolog.
The logic programming paradigm use by Prolog is something I have not learned about before. The development of axioms and problems solving by querying the axioms is the foundation of languages such as prolog. The engine of Prolog is a backward chaining theorem prover. The axioms in logic programming need to be Horn clauses: disjunctions of literals with exactly one positive literal

An example:

1
2
3
4
5
6
7
king(X) &amp; greedy(X) → evil(X).

king(john).

greedy(john).

?evil(john).

In the  tutorial we were able to do some basic playing with a toy implementation by Lloyd Allison:

http://www.csse.monash.edu.au/˜lloyd/tildeLogic/Prolog.toy/

Prolog relies very heavily on unification, a process that we were actually unable to correctly re-inact in the tutorial.

p(X, c(X,Y)).
p(X, c(Y,Z)) <= p(X,Z)
?p(A, c(1,c(2,c(3,nil)))).

Prolog Answer:

p(1, c(1, c(2, c(3, nil)))) yes
p(2, c(1, c(2, c(3, nil)))) yes
p(3, c(1, c(2, c(3, nil)))) yes

After reading the tutorial solution, I am not really much clearer on the proves for each of these outcomes. I will have to follow up in the lecture.

NLP

We discussed the surface level methodologies for NLP:

  • Lexical analysis
  • Syntactic analysis
  • Semantic analysis
  • Pragmatic analysis

The focus of the lecture was however on the limitations of NLP. How ambiguity of words, their meaning and context makes effective NLP very difficult. Implications were another issue for NLP covered for some time.

Next came some approaches for overcoming the challenges of NLP. Statistical approaches such as N-Gram analysis. This veered the lecture into information retrieval , discussing the techniques used by search engines such as google to interpret searches.

On the topic of NLP I wondered if there were any large knowledge bases being assembled to try and assist in the task. Yahoo have a cluster of computers working on this:

http://en.wikipedia.org/wiki/Never-Ending_Language_Learning
Another interesting project:
http://en.wikipedia.org/wiki/Wolfram_Alpha