When developing an application for Android you will require additional resources. This data is kept in resource files. The resource files live under the /res directory and consist of the following items:

• Colors
• Strings
• Dimensions
• Drawables (Images and Icons)
• Sounds
• Videos
• Data Files
• Layout Files

The Rules for resource files are:

• Must be lowercase
• May contain: letters, numbers, underscores and periods
• Filenames must be unique
• XML Name attributes must be unique as well

To reference the resources from a compiled resource you would use something like the following format depending on the type of resource:

String app_name = getResources().getString(R.string.app_name);

The template for reading resources is either:
Local

@[resource type]:/[resource name]

or
Android

@android:[resource type]:[resource name]

or from a layout file:

@string/app_name

or from the android system resources:

@android:string/ok

Tags: android

When working with Hadoop you might need to move files in and out of the HDFS. The hadoop fs commands can get tedious to type out over and over again, so I came up with some aliases that work to reduce my typing.

The commands that I have found myself using most often are:

Listing the directories and files:

hadoop fs -ls <filename(s)>

hls <filename(s)>

Viewing the contents of files:

hadoop fs -cat <filename(s)>

hcat <filename(s)>

I usually pipe that into an awk script to verify data.

Removing Entries from the HDFS:

hadoop fs -rm <filename(s)>

hrm <filename(s)>

Copying files to HDFS:

hadoop fs -copyFromLocal <local filename> <target directory or filename>

hcpto <local filename> <target directory or filename>

Copying from HDFS to the local system:

hadoop fs -copyToLocal <remote file(s)> <target directory or filename>

hcpfrom <remote file(s)> <target directory or filename>

Note: The <filename(s)> should be replaced with whatever files or directories including using the ‘*’ to select multiple files. Only the copy commands should have a singular entry for the target directory.

Here are the alias entries I use:

alias h='hadoop fs'
alias hcat='hadoop fs -cat '
alias hls='hadoop fs -ls '
alias hrm='hadoop fs -rm '
alias hcpto='hadoop fs -copytoLocal'
alias hcpfrom = 'hadoop fs -copyFromLocal'

Tags: hadoop

I have been enjoying learning Hadoop and have had to debug issues within a current process job to enhance it for new data points. It is quite the challenge to debug a distributed system but I have found two ways to get some meaningful input from the system. One way is using Counters and the other is to use MultipleOutputs to capture output from errors(This will be in a later post). This article will show how counters can be used.

The Counter method allows you to specify a set of enum values to specify the counter name:

public enum Counters {
  CHOCOLATE,
  VANILLA,
  MINT_CHOCO_CHIP
}

This will be used as the identifier for the counter that is in use. The same code can be used within the Mapper or the reducer depending on where you are looking for the counts. The difference is where in the final output of the Hadoop process that the counts show up.

if (flavor.contains("CHOCOLATE")) {
    context.getCounter(Counters.CHOCOLATE).increment(1);
}
if (flavor.contains("VANILLA")) {
    context.getCounter(Counters.VANILLA).increment(1);
}
if (flavor.contains("MINT_CHOCO_CHIP")) {
    context.getCounter(Counters.MINT_CHOCO_CHIP).increment(1);
}

This way you will find a line for each counter that was incremented during the Hadoop process.

I have used it to highlight problems with in the code or just to make sure that a segment of code has been run.

Tags: BigData, hadoop

Variables in Python are declared on first use.
Variable names in Python must:

• Start with a letter or an underscore character
• Not start with a number
• Only contain alpha-numeric characters and underscores (A-Z,a-z, 0-9 and _)
• Be case sensitive (ex. name and Name are different variables)

Good variable names

apple
b_funky
_tester

The underscore as the first character usually indicates that the function is for internal use. This convention is mentioned in PEP 8.

Bad variable names

8million
$dog
+happy

The scope of a Python variable depends where it is defined.

If it defined outside of any function or class, then that variable is global. This means that all functions can access that variable.

some_var= "Eric"

If it is defined within a class or function, than that variable is considered to have local scope.

def print_name(name):
    greeting = "Good Morning,"
    print(greeting,name)

The variable some_var is global and can be used throughout the program, but the variables name and greeting are local to the function and are not available outside of the function.

Tags: Python

If you want to get your hands on Crypto Currency but are not willing to put up cash up front, there are a collection of sites out there that are called faucets. These faucets “drip” crypto currencies. This will not make you rich today but it will give you a foothold into the crypto currency domain.

Tags: bitcoin, cryptocurrency, dogecoin, litecoin