I have been using dataclasses in my python data retrieval scripts but on the occasions when a field is added to the API return I needed a way to make the scripts continue to run. The added benefit of notifying the script user about any unknown fields helps too.

The following code with a Warning message to show any fields that are not predefined in the dataclass.

from dataclasses import dataclass
from dataclasses import fields

def filter_unexpected_fields(cls):
    original_init = cls.__init__

    def new_init(self, *args, **kwargs):
        expected_fields = {field.name for field in fields(cls)}
        cleaned_kwargs = {key: value for key, value in kwargs.items() if key in expected_fields}
        unknown_kwargs = {key: value for key, value in kwargs.items() if key not in expected_fields}
        original_init(self, *args, **cleaned_kwargs)
        # Handle unknown fields (optional)
        if unknown_kwargs:
            # Log a warning or raise an exception based on your needs
            print(f"Warning: Unknown fields found: {unknown_kwargs}")
    cls.__init__ = new_init
    return cls

@filter_unexpected_fields
@dataclass
class Application:
     id: int
     name: str

a1 = { 'id': 1, 'name': 'WebGoat',"runs":5}
app = Application(**a1)

So given the following code, the “runs” field is not defined in the dataclass object, but with the “@filter_unexpected_fields” decorator, the code will continue to work but the additional warning message will notify about any undefined fields.
_{¹ based on answer on this page: https://stackoverflow.com/questions/54678337/how-does-one-ignore-extra-arguments-passed-to-a-dataclass}

I recieved a new laptop and the Python certificates were failing for me. The quickest, but not necessarily the most secure method for fixing this is to either use the –trusted-host entry for each host that throws a certificate exception or to create a file in the ~/.config/pip/pip.conf file like the one below. This covers the main sites that pip uses to install packages.

[global]
trusted-host = pypi.python.org
               pypi.org
               files.pythonhosted.org

I was writing a Meeting tracking app in Laravel and using the shanmuga3/laravel-entrust module for roles and permissions. I created a new role for the application but needed to support some superadmin functionality within the site but was having issues with some of the routing. I found that padding the superadmin and admin with a pipe delimited list directly to the ‘role:’ entry worked.

Route::group(['prefix' => 'admin','middleware' => ['role:superadmin|admin']], function() {

        Route::get('/settings', function() {
          return "a";
        })->name('admin.edit');
})
;

I was working on my Android app skills and was creating a simple calculator. It was very basic and worked great until I started to play with division. I thought, this will be easy. Just type, 3 / 2 = and I would be given the answer. We all know the answer is, of course, 1.49999998.

Wait a minute! Why would this be the answer you ask? This has to do with the data type used and if you are doing calculations where the result really matters such as for money, then you should not be using a double or a float type.

BigDecimal to the rescue. The BigDecimal type “fixes” the issues that come up when you are using decimal numbers and stop the precision loss. This means that the 3 / 2 = will indeed give you 1.5. But you need to use the BigDecimal functions to add, subtract, multiply and divide. Not just the ‘+’ (plus), ‘-‘ (minus), ‘*’ (multiply), ‘/’ (divide) operators.

Instead of using doubles, like this:

double firstNumber = 3;
double secondNumber = 2;

You should use BigDecimal, like this:

BigDecimal firstNumber = new BigDecimal(3);
BigDecimal secondNumber = new BigDecimal(2);

But now when you try to add, subtract, multiply or divide using a the traditional operators, it no longer works. You will need to replace them with the methods named after the operations you are attempting.

// Add 
double addResult = firstNumber + secondNumber;  //error
BigDecimal addResultBd = firstNumber.add(secondNumber);
// Subtract 
double subtractResult = firstNumber - secondNumber; //error
BigDecimal subtractResultBd = firstNumber.subtract(secondNumber);
// Multiply
double multiplyResult = firstNumber * secondNumber; //error
BigDecimal multiplyResultBd = firstNumber.multiply(secondNumber);
// Divide
double divideResult = firstNumber / secondNumber; //error
BigDecimal divideResultBd = firstNumber.divide(secondNumber);

So using BigDecimal can save your sanity when your calculations are critical to what you are expecting. Calculators that give you a “close enough” representation of the numbers you are expecting are not very good.

I was creating a new program and wanted to generate an output string that showed the count of certain types of entries. When I attempted the simple output to get “Number of rows: 5” by using the following:

Incorrect call

rowcount = 100
print("Number of Rows: " + rowcount) 

I got the following error “TypeError: can only concatenate str (not “int”) to str” . This meant that the type of rowcount was an int but needed to be a string. There is a handy function call that takes the int and converts it to a string. The function is str() and it converts the int argument to a string value.

rowcount = 100
print("Number of Rows: " + str(rowcount))

There are other ways to output a number such as ‘%s’, format and f-string. But for my purposes. ‘str’ gave me the debugging info I needed.

Tags: Python

There are times that you may want to assign a different value to a variable based on the value of another variable. This can be accomplished in many compiled languages with the ternary operator.

In Python the syntax for this is different because python was written to allow you to read a line of code and have a better understanding of what the code is doing. Python uses conditional assignment where you can use an if statement to help decide what the variable should be assigned.

a = "Happy" if weather == "sunny" else "Sad"

So reading this code states: “a is set to “Happy” if weather equals sunny else a is “Sad”” .

When developing an application for Android you will require additional resources. This data is kept in resource files. The resource files live under the /res directory and consist of the following items:

• Colors
• Strings
• Dimensions
• Drawables (Images and Icons)
• Sounds
• Videos
• Data Files
• Layout Files

The Rules for resource files are:

• Must be lowercase
• May contain: letters, numbers, underscores and periods
• Filenames must be unique
• XML Name attributes must be unique as well

To reference the resources from a compiled resource you would use something like the following format depending on the type of resource:

String app_name = getResources().getString(R.string.app_name);

The template for reading resources is either:
Local

@[resource type]:/[resource name]

or
Android

@android:[resource type]:[resource name]

or from a layout file:

@string/app_name

or from the android system resources:

@android:string/ok

Tags: android

I have been enjoying learning Hadoop and have had to debug issues within a current process job to enhance it for new data points. It is quite the challenge to debug a distributed system but I have found two ways to get some meaningful input from the system. One way is using Counters and the other is to use MultipleOutputs to capture output from errors(This will be in a later post). This article will show how counters can be used.

The Counter method allows you to specify a set of enum values to specify the counter name:

public enum Counters {
  CHOCOLATE,
  VANILLA,
  MINT_CHOCO_CHIP
}

This will be used as the identifier for the counter that is in use. The same code can be used within the Mapper or the reducer depending on where you are looking for the counts. The difference is where in the final output of the Hadoop process that the counts show up.

if (flavor.contains("CHOCOLATE")) {
    context.getCounter(Counters.CHOCOLATE).increment(1);
}
if (flavor.contains("VANILLA")) {
    context.getCounter(Counters.VANILLA).increment(1);
}
if (flavor.contains("MINT_CHOCO_CHIP")) {
    context.getCounter(Counters.MINT_CHOCO_CHIP).increment(1);
}

This way you will find a line for each counter that was incremented during the Hadoop process.

I have used it to highlight problems with in the code or just to make sure that a segment of code has been run.

Tags: BigData, hadoop

Variables in Python are declared on first use.
Variable names in Python must:

• Start with a letter or an underscore character
• Not start with a number
• Only contain alpha-numeric characters and underscores (A-Z,a-z, 0-9 and _)
• Be case sensitive (ex. name and Name are different variables)

Good variable names

apple
b_funky
_tester

The underscore as the first character usually indicates that the function is for internal use. This convention is mentioned in PEP 8.

Bad variable names

8million
$dog
+happy

The scope of a Python variable depends where it is defined.

If it defined outside of any function or class, then that variable is global. This means that all functions can access that variable.

some_var= "Eric"

If it is defined within a class or function, than that variable is considered to have local scope.

def print_name(name):
    greeting = "Good Morning,"
    print(greeting,name)

The variable some_var is global and can be used throughout the program, but the variables name and greeting are local to the function and are not available outside of the function.

Tags: Python

When I develop in the web, there are plenty of times that I want to see information about where I am in the program or what is going on in the Javascript when I load the page. I use the Console to watch updates but it can be difficult to read when you have possibly hundreds of log statements.
Usually you can just use the standard console.log function:

> console.log("Some information");
Some information
or 

> console.debug("Some information");
Some information

or 

> console.info("Some information");
Some information

You also get the line number and file that the output comes from in the console.

But did you know that you can use the following:

console.clear()
console.error(object)
console.table(array)
console.warn()

Let’s say you want to clear the current console. You can place in your code, or use it within the console itself.

console.clear();

Or you can get color coded lines with the warn or error functions:

> console.warn("This is a warning");

> console.error("Error: Danger");

And if you need to display an array that and want a better layout use console.table():
> console.table(['a','b','c']);