Tuesday, 31 July 2012

Partial & Tagged Literals

Partial

Partial is a great function. The doc (doc partial) describes it as taking function f and fewer than the normal arguments to f. What is returned is a function that a variable number of remainder args. There are many potential applications and allow you to essentially create duplicate but similar functions without code duplication.

(def only-strings (partial filter string?))
(only-strings ["first test" :anothertest 3 4 "last test"])
("first test" "last test")

Tagged Literals

I mentioned a tagged literal in my last post. Namely the #inst "2012" part. That convenience is a tagged literal for creating a date. I accidently noticed there's a new one in the build i've done from github. That's the #uuid which seems to have appeared from 1.4-alpha4. It appears there aren't many available beyond that yet but they're definitely worth keeping an eye on as they're very handy. Hopefully at some point i'll do a post on creating your own.

Sunday, 29 July 2012

Focusing on Clojure

Back on Numbers

Numbers can be appended with letters that allow you to specify the type.

(type 1000)
java.lang.Long

(type 1000N)
clojure.lang.BigInt

(type 1000M)
clojure.lang.BigDecimal

Map, Reduce & Filter

These healthy staples allow you to apply functions to sequences. The usage is dependant on context. I'll go through each one:

Map

Map applies a function to a sequence of data and returns a sequence of the same length with the function applied to each

(map (fn [n] (* n 10)) '(1 2 3 4 5 6 7 8 9 10))
(10 20 30 40 50 60 70 80 90 100)

Functions with multiple arity can also be used. The result of this function is multiplying arg 1 with arg 2. This has given us a list of square numbers.

(map (fn [n m] (* n m)) (range 1 11) (range 1 11))
(1 4 9 16 25 36 49 64 81 100)

Reduce

Reduce is similar however instead of returning a list of the same size, the goal is to get to a single result. So arg 1 and arg 2 are used in combination to produce a single result, which then becomes arg 1 and #3 becomes arg 2. This is best shown with an example.

(reduce * (range 1 6))
120

To produce that output the following is equivalent:
((((1 * 2) * 3) * 4) * 5) * 6 = 120

Filter

Filter is again similar, however the size of the output is variable. You could end up with a list as long as the original or empty or anywhere in between. Filter works by providing a predicate function with the list and outputs the values of the list where the function returns a truthy value.

(filter even? (range 1 11))
(2 4 6 8 10)

ArrayList

Since clojure is on the JVM, it can work with java code seemlessly

(def x (java.util.ArrayList.))
(map #(.add x %) (range 1 11))
(true true true true true true true true true true)
x
#<ArrayList [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]>

Sequential destructuring

Vectors can be sequentially destructured as demonstrated here

(def x [5 10 [3 5] 99.9])
[5 10 [3 5] 99.9]
(let [[a b _] x] (+ a b))
15

The & can also be used

(let [[a & rest] x] rest)
[10 [3 5] 99.9]

Date

Just as a quick reference, #inst "" can be used for quick date creation

(def d #inst "2012")
#inst "2012-01-01T00:00:00.000-00:00"
(type d)
java.util.Date

Repeatedly Partial

Lookup these values

(take 15 (repeatedly (partial rand-int 60)))
(48 18 41 4 41 49 45 22 0 25 17 57 45 36 52)

Others

Some other useful functions found are memoize, doseq, dotimes, for, range, doc, find-doc and source.

Thursday, 26 July 2012

Clojure revisit

Well, it appears i've forgotton all my clojure knowledge. The only thing I appear to have remembered is the thinking style behind it all. So, after reviewing some basics today let's write them down.

Getting to the REPL

java -cp clojure-1.4.0.jar clojure.main

Basic Math

(+ 1 2 3)
(- 3 2 1)
(/ 1 2 3)
(* 1 2 3)

Numbers

Numbers can be represented infinitely appended with an M or as ratio's.
2r10110110 ; this is radix 2 (bin)
8r755 ; this is radix 8 (oct)
16rFFF ; this is radix 16 (hex)

Types

There are many forms of types available in clojure
nil ; equivalent to null essentially, it is neither true or false
true false ; these are boolean values :fred ; this is a symbol
\c \newline \space \tab ; these are chars
"string" ; this is a string literal
(1 2 3) ; this is a list
[1 2 3] ; this is a vector
{:a 1 :b 2 :c 3} ; this is a map
#{:a :b :c} ; this is a set
^{:a 1}[1 2 3] ; this is a vector with metadata

Functions

Functions are defined with the fn function, however this creates anonymous functions and to store them you need to use defn
(defn average [x y] (/ (+ x y) 2))

Useful functions

Just some I remember from last time

  • (zipmap [:a :b :c] [1 2 3])
    {:c 3, :b 2, :a 1}
  • (doc zipmap)
    ; prints current documentation
  • (source zipmap)
    ; prints source
  • (cons :y [:a :b :c])
    [:y :a :b :c]

In my next post about clojure I want to include some stuff about map, reduce and filter.

Friday, 13 July 2012

Starting an new Activity

Before starting a new activity from a current one, you need to setup the classes.

Intent display = new Intent(this, DisplayMessageActivity.class);
EditText editText = findViewById(R.id.edit_text);
String message = editText.getText().toString();
display.putExtra(EXTRA_MESSAGE, message);
startActivity(display);

From the DisplayMessageActivity then, you can get the extra like this.

Intent intent = getIntent();
String message = intent.getStringExtra(PreviousActivityClassname.EXTRA_MESSAGE);

Thursday, 12 July 2012

Android UI Weight

Android weights are like drink mixes, "2 parts vodka, 1 part coffee liqueur". If you set one widget to a weight of 2 and another to a weight of 1 - the first will get 2/3rds of the space and the other will get 1/3rd.
When using weights, it's a good idea to set the layout_width to 0 to improve performance. It improves performance because it doesn't need to work out something that ultimately is irrelevant.

Tuesday, 10 July 2012

Design Patterns

Some I can think of off the top of my head:

  • Subscriber
  • Builder
  • Singleton
  • Adapter
  • Factory
  • Facade

Patterns can be broken down into five types, four of which were described in gang of four. The fifth has been added later but its contents predate many of the other patterns. They are
  1. Creational Patterns
  2. Structural Patterns
  3. Behavioural Patterns
  4. Concurrency Patterns
  5. Architectural Patterns

Creational Patterns

So this includes Abstract Factory, Builder, Factory Method, Lazy Initialisation, Multiton Pattern, Object Pool, Prototype and Singleton

Structural Patterns

Adapter, Bridge, Composite, Decorator, Facade, Front Controller, Flyweight, Proxy and Module

Behavioural Patterns

Blackboard, Chain of Responsibility, Command, Interpreter, Iterator, Mediator, Momento, Null Object, Observer, Servant, Specification, State, Strategy, Template method and Visitor

Concurrency Patterns

Active Object, Balking, Message Design, Double-checked Locking, Event-based asynchronous, Guarded Suspension, Lock, Monitor Object, Reactor, Read-Write Lock, Scheduler, Thread Pool and Thread-Specific Storage

Architectural Patterns

SOA, MVC, N-Tier, Event Driven, Peer-to-Peer et al

As each pattern could be its own post, i won't attempt to describe them here but will work through the list as individual posts

Monday, 9 July 2012

Makefile

Before we can have a useful makefile, we should first create some files that the build tool can be run against.

module.h

#include

void sample_func();

module.c

#include "module.h"

void sample_func() {
    printf("Hello World!");
}

main.c

#include "module.h"

void sample_func();

int main() {
    sample_func();
    return 0;
}

So, now we have a directory with 3 files. A module which includes a header and source file. And a main source file that includes an entry point and has a dependency on our module. Let's make a Makefile for this.

GCC = gcc
FLAGS = -O2 -j3
CC = ${GCC} ${FLAGS}

all: main.o module.o
  ${CC} main.o module.o -o target_bin
main.o: main.c module.h
  ${CC} -I . -c main.c
module.o: module.c module.h
  ${CC} -I . -c module.c
clean:
  rm -rf *.o
  rm target_bin

Sunday, 8 July 2012

Android UI basic

Basic LinearLayout

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:orientation="vertical"
    android:layout_width="fill_parent"
    android:layout_height="wrap_content" >
</LinearLayout>


When this code is saved, the ADK will use the Android Asset Packaging Tool (AAPT) to generate R.java

In Android, a widget is called a View and a layout container like LinearLayout is called a ViewGroup. Above has created an empty ViewGroup

Asking a Question

In order to ask a question, we'll need to add a TextView to the top of the layout. To add this, put the following in between the LinearLayout element.

<TextView
    android:id="@+id/question"
    android:layout_width="fill_parent"
    android:layout_height="wrap_content"
    android:text="Please wait..."
  />

Now for the answers we're going to want multiple choices. If we use buttons for starters that'll do, but since we can't set a variable number of buttons we'll need to contain them. So let's create a new LinearLayout.

<LinearLayout 
    android:id="@+id/answers"
    android:orientation="vertical"
    android:layout_width="fill_parent"
    android:layout_height="wrap_content" >
</LinearLayout>

Then, for each button we want something similar to this:

<Button
    android:id="@+id/yes"
    android:text="Yes!"
    android:layout_width="fill_parent"
    android:layout_height="wrap_content"
/>

Saturday, 7 July 2012

Terminal Stuff

ffmpeg -i input.mp4 -c:v copy -c:a copy output.mkv

ffmpeg takes an input file, copies the codec for video and audio and sets the output file and format


openssl enc -aes-256-cbc -salt -in file.tar.gz > outfile.tar.gz

openssl set to encryption mode with aes-256-cbc encryption algorithm and salted. Sets the input file and directs the output to a file


expand -t4 Tabs.java

expand sets a number of spaces to replace tabs with and an input file, outputs to stdout


find . -iname \*\.epub -exec mv {} ~/epub/ \;

finds files using the current directory as the root, filters on the case insensitive name a regex pattern of *.epub and for each find executes mv the result to ~/epub/


nmap -sTUV -O -p- -PN 192.168.1.*

nmap set to TCP and UDP and versioning mode, also set to identify an operating system. Will scan all ports and behave as if the host is up regardless of responses. Will scan a class C network for all ip's between 192.168.1.1-255


:%s/search_regex/replacement_text/g

Vim command for finding and replacing text, the G flag represents globally across the whole file. Otherwise it will be limited to the current line


Monday, 2 July 2012

Virtual Machine Options

As Virtualisation makes it relatively easy to setup a large number of systems, it can help you configure a large number of systems. Each dedicated to a specific service. To that end, it can be separated into 5 different categories.

  • Application Level
  • Platform-level VMs
  • Paravirtualization
  • Hardware-assisted Virtualisation
  • Bare-metal Virtualisation


Application Level


Systems like WINE are application level virtualisations. They allow the installation of a single application that were designed to be run on an entirely different system.

Platform-level VMs


Applications such as VirtualBox or VMware Player are platform-level VMs. They emulate systems to allow for contained systems to have their own hardware that systems can be installed against.

Paravirtualization

While similar to Platform-level VMs, they work with fewer resources and usually require a specialised kernel such as Xen.

Hardware-assisted Virtualisation


A hardware interface where VMs have access to hardware features of the CPU such as VMX/SVM flags on Intel/AMD chips respectively.

Bare-metal Virtualisation


Some VM systems include a minimal operating system dedicated to VM operation. Two examples of this are Citrix XenServer and VMware ESX.

Notes


The KVM solution that comes with RHEL6 is known as hypervisor, a VM monitor that supports the running of multiple operating systems concurrently on the same CPU. KVM replaces the previous default of Xen.

KVM has replaced Xen in many open-source operating systems. XenSource which is owned by Citrix started working with Microsoft after RHEL5 was released.

Hard Drives and Linux (snippet)

SATA or Serial Advanced Technology Attachment


Systems now with port multipliers, can easily be configured to have 16 drives through SATA. Older systems could only handle 4 through PATA (Parallel Advanced Technology Attachment).

SCSI or Small Computer Systems Interface


Depending on the SCSI hardware, you could have up to 31 hard drive devices connected at a time.

Pick 'n' Mix


While you can combine these in as many configurations as your hardware will allow, the Linux boot files from /boot/ works on only one of the first two hard drives. If Linux is installed on the second drive, you'll need other media recognised by the BIOS/UEFI to get access to it.

The caveat to this however is that Linux doesn't have to be installed to a local drive. There are options that allow it to be placed on SAN (Storage Area Network), DASD (Direct Access Storage Device), hardware RAID devices (Redundant Array of Independent Disk) and more.

Access Control System Considerations

Before implementing any kind of access control system, the security professional needs to consider potential vulnerabilities in the proposed system. The reason for needing to consider these is because they are what give rise to threat. Threats must also be considered because they lead to risks.

Risk is the potential that a vulnerability may be exploited.

Biometrics / Biometric Authentication

Biometrics come in many forms and refers to the identification of humans by their characteristics or traits. It is used as a form of identification and access control.
They can all be broken down into two categories: Physiological or Behavioural. The latter is also sometimes known as Behaviometrics. Some examples of them are (in order of effectiveness)
  1. Palm Scan
  2. Hand Geometry
  3. Iris Scan
  4. Retina Scan
  5. Fingerprint

Effectiveness is not a sole factor to consider when choosing biometric systems however. There are many more things that need to be weighed in. Performance is generally measured by the following methods:
  • False Accept Rate (FAR) or False Match Rate (FMR)
  • False Reject Rate (FRR) or False Non-Match Rate (FNMR)
  • Relative Operating Characteristics (ROC)
  • Equal Error Rate (EER) or Crossover Error Rate (CER)
  • Failure to Enrole Rate (FER)
  • Failure to Capture Rate (FCR)
  • Template Capacity

Android Services

A Service is a process that runs in the background, without interacting with the user. Android provides some pre-defined services which can be accessed through getSystemService().

To declare your own service, it must be defined in the manifest as a service and the implementing class must extend Service or one of its subclasses. A service will not automatically run in its own thread without the process attribute. They by default run in the main thread of the hosting process. To define the process, in the manifest set the process under the application tab. An example is :my_service. The colon at the start tells android that the process is private to declaring applications. Otherwise the process would be globally accessible.

An Activity can start the Service by calling the startService() method and stop the service via the stopService() method. After a service is started the onCreate() method is called and later the onStartCommand gets invoked with the Intent.

Starting services automatically


To start services automatically after boot involves registering a BroadcastReceiver to the Android android.intent.action.RECEIVE_BOOT_COMPLETED system event. This requires the RECEIVE_BOOT_COMPLETED permission. In the broadcast receiver class, override the onReceive() method and call the service like:

Intent service = new Intent(context, MyService.class); context.startService(service);

Related Information

Android Networking
Android Services

MBeans

Standard MBeans

A standard MBean is created by creating an interface that complies with the naming convention of SomethingMBean. The implementation is then called Something. A NotCompliantMBeanException exception will be thrown if this convention is not followed.
Following the naming convention, the standard JavaBean conventions can be followed in terms of getters and setters.

Remote Monitoring

Once the resource has been created by MBeans, the management of that resource is done by a JMX agent. The core component of the JMX agent is the MBean server. The MBean server is a managed object server in which MBeans are registered. A JMX agent also includes a set of services used to manage MBeans.
To create a basic JMX Agent, the following is minimal

MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
ObjectName on = new ObjectName("package.location.of.mbeans:type=Example");
Example e = new Example();
mbs.registerMBean(e, on);

Provided the application remains running, you can then fire up jconsole and monitor the process within there. Under the MBeans tab you should then find package.location.of.mbeans with a unit below it called Example which can then be observed, called, monitored etc.

Warning

I had some issues getting the small example working on a windows machine. This ended up being a combination of rights access (which I was struggling with on windows) and folders that existed in %TMP%. They had my username including capitals which is correct as far as I know, but they needed deleting and when they were recreated with lowercase they worked fine. You can see evidence of this by having jconsole grayed out on its monitoring processes. There is more on this here.

I also had issues where i couldn't connect to the weblogic process. This turned out to be a classpath problem which can be solved by using the jconsole of weblogics jrocket jvm. More than on that here

MXBeans

An MXBean is a type of MBean that references only a predefined set of data types. This allows you to be more flexible in allowing your clients (including remote clients) to use your MBean without allowing them access to more data than they need. The process for creating them is similar to MBeans.

First you must create an interface that either follows the naming convention of SomethingMXBean, or annotate the interface with @MXBean. Then have a class implement that interface. Unlike MBeans, there is no requirement on the implementation to be called Something.

The idea behind MXBeans is that types such as MemoryUsage that are referenced in the MXBean interface MemoryMXBean are mapped to a set of types known as Open Types that are defined in the package javax.management.openmbean. The general principle is for simple types such as int or String to remain unchanged, while complex types such as MemoryUsage get mapped to the standard type CompositeDataSupport.

MXBeans Example

To create an MXBean interface, almost exactly the same as an MBean example

public interface QueueSamplerMXBean {
    public QueueSample getQueueSample();
    public void clearQueue();
}


Then the implementation:

public class QueueSampler implements QueueSamplerMXBean {

    private Queue queue;

    public QueueSampler(Queue queue) {
        this.queue = queue;
    }

    public QueueSample getQueueSample() {
        return new QueueSample(new Date(), queue.size(), queue.peek());
    }

    public void clearQueue() {
        queue.clear();
    }
}


Then unlike MBeans, we need an additional class for the type.

public class QueueSample {

    private final Date date;
    private final int size;
    private final String head;

    @ConstructorProperties({"date", "size", "head"})
    public QueueSample(Date date, int size, String head) {
        this.date = date;
        this.size = size;
        this.head = head;
    }

    public Date getDate() {
        return date;
    }

    public int getSize() {
        return size;
    }

    public String getHead() {
        return head;
    }
}


The MXBean framework calls all the getters in QueueSample to convert the given instance into a CompositeData instance. It uses the @ConstructorProperties annotation to reconstruct a QueueSample instance from a CompositeData instance.

Registering the MXBean is much the same as registering an MBean, however in this case the difference is we need to construct the queue system before we can use it. We register the QueueSampler class against the MBean server and don't actually reference the returned type in our calling code.

Notifications

The model for adding notification support is actually quite simple. When you create the implementation of your MBean, you will want to extend somewhere in your hierarchy the class NotificationBroadcasterSupport and where you want to create notifications simple instantiate a subclass of Notification such as the pre-existing AttributeChangeNotification. Once constructed it can be passed to sendNotification().

After adding this support, you will have the ability to subscribe to notifications from tools such as jconsole.

Related resources

Oracle JMX Trail
wiki MBeans
MBean Server interface