Python: Dealing with unknown fields in dataclass

digg del.icio.us TRACK TOP
By Eric Downing | Filed in Programming, Python | No comments yet.

I have been using dataclasses in my python data retrieval scripts but on the occasions when a field is added to the API return I needed a way to make the scripts continue to run. The added benefit of notifying the script user about any unknown fields helps too.

The following code with a Warning message to show any fields that are not predefined in the dataclass.

from dataclasses import dataclass
from dataclasses import fields

def filter_unexpected_fields(cls):
    original_init = cls.__init__

    def new_init(self, *args, **kwargs):
        expected_fields = {field.name for field in fields(cls)}
        cleaned_kwargs = {key: value for key, value in kwargs.items() if key in expected_fields}
        unknown_kwargs = {key: value for key, value in kwargs.items() if key not in expected_fields}
        original_init(self, *args, **cleaned_kwargs)
        # Handle unknown fields (optional)
        if unknown_kwargs:
            # Log a warning or raise an exception based on your needs
            print(f"Warning: Unknown fields found: {unknown_kwargs}")
    cls.__init__ = new_init
    return cls

@filter_unexpected_fields
@dataclass
class Application:
     id: int
     name: str

a1 = { 'id': 1, 'name': 'WebGoat',"runs":5}
app = Application(**a1)



So given the following code, the “runs” field is not defined in the dataclass object, but with the “@filter_unexpected_fields” decorator, the code will continue to work but the additional warning message will notify about any undefined fields.
1 based on answer on this page: https://stackoverflow.com/questions/54678337/how-does-one-ignore-extra-arguments-passed-to-a-dataclass

Python: pip certificate failure

digg del.icio.us TRACK TOP
By Eric Downing | Filed in Python, Scripting | No comments yet.

I recieved a new laptop and the Python certificates were failing for me. The quickest, but not necessarily the most secure method for fixing this is to either use the –trusted-host entry for each host that throws a certificate exception or to create a file in the ~/.config/pip/pip.conf file like the one below. This covers the main sites that pip uses to install packages.

[global]
trusted-host = pypi.python.org
               pypi.org
               files.pythonhosted.org

I was writing a Meeting tracking app in Laravel and using the shanmuga3/laravel-entrust module for roles and permissions. I created a new role for the application but needed to support some superadmin functionality within the site but was having issues with some of the routing. I found that padding the superadmin and admin with a pipe delimited list directly to the ‘role:’ entry worked.

Route::group(['prefix' => 'admin','middleware' => ['role:superadmin|admin']], function() {

        Route::get('/settings', function() {
          return "a";
        })->name('admin.edit');
})
;

Zelda – Tears of the Kingdom – Teniten Shrine Playthrough video

 1,338 total views

Tags: , ,

Node: Fix line endings by platform

digg del.icio.us TRACK TOP
By Eric Downing | Filed in Linux, Node, Windows | No comments yet.

Using a Node script to query a database to do corrections was working fine on my local Windows based system until we tried to automate the process. We moved it to a Jenkins server which happened to be on a Linux system and the script stopped working. After some debugging it turned out that the way we were attempting to differentiate data returns from the queries depended on the line ending.

On a Windows system, the line endings are ‘\r\n’ or carriage return, new line. This had worked well while we were running the script locally, but once it went to the Linux system the line endings were just a ‘\n’ or new line. The line ending was not being matched so this led us to figure out how to tell what platform was being used. The process.platform variable contains the platform name.

The process.platform returns ‘win32’ for Windows, ‘darwin’ for MacOS and ‘linux’ for linux systems. There are other platforms supported but these are all that mattered to us. Using this variable we were able to replace the line endings and the data started working as expected on all our platforms.

The following code snippet sets the lineendings for our sql code:

let platform = process.platform;
let lineendings = '\n';
if (platform == 'win32') {
    lineendings = '\r\n';
}

Tags: ,

Laravel on windows: Private key does not exist

digg del.icio.us TRACK TOP
By Eric Downing | Filed in Frameworks, Laravel | No comments yet.

I was setting up my Laravel development using the Homestead on a new laptop and my usual tool chain includes installing git and using git bash. When I tried the ‘vagrant up’ command I got the following error:

Check your Homestead.yaml (or Homestead.json) file, the path to your private key does not exist.

The default location for that file is the .ssh directory within your home directory. This directory does not exist unless you have run the ssh-keygen tool to create it. Run the following command to generate the key file. For local development you can just use the default options.

ssh-keygen -t rsa -C "you@homestead"

Once the file is generated you can run ‘vagrant up’ and continue your development.

Java: Fix floating point math

digg del.icio.us TRACK TOP
By Eric Downing | Filed in Java, Programming, Useful | No comments yet.

I was working on my Android app skills and was creating a simple calculator. It was very basic and worked great until I started to play with division. I thought, this will be easy. Just type, 3 / 2 = and I would be given the answer. We all know the answer is, of course, 1.49999998.

Wait a minute! Why would this be the answer you ask? This has to do with the data type used and if you are doing calculations where the result really matters such as for money, then you should not be using a double or a float type.

BigDecimal to the rescue. The BigDecimal type “fixes” the issues that come up when you are using decimal numbers and stop the precision loss. This means that the 3 / 2 = will indeed give you 1.5. But you need to use the BigDecimal functions to add, subtract, multiply and divide. Not just the ‘+’ (plus), ‘-‘ (minus), ‘*’ (multiply), ‘/’ (divide) operators.

Instead of using doubles, like this:

double firstNumber = 3;
double secondNumber = 2;

You should use BigDecimal, like this:

BigDecimal firstNumber = new BigDecimal(3);
BigDecimal secondNumber = new BigDecimal(2);

But now when you try to add, subtract, multiply or divide using a the traditional operators, it no longer works. You will need to replace them with the methods named after the operations you are attempting.

// Add 
double addResult = firstNumber + secondNumber;  //error
BigDecimal addResultBd = firstNumber.add(secondNumber);
// Subtract 
double subtractResult = firstNumber - secondNumber; //error
BigDecimal subtractResultBd = firstNumber.subtract(secondNumber);
// Multiply
double multiplyResult = firstNumber * secondNumber; //error
BigDecimal multiplyResultBd = firstNumber.multiply(secondNumber);
// Divide
double divideResult = firstNumber / secondNumber; //error
BigDecimal divideResultBd = firstNumber.divide(secondNumber);

So using BigDecimal can save your sanity when your calculations are critical to what you are expecting. Calculators that give you a “close enough” representation of the numbers you are expecting are not very good.

Freebitco.in Review

digg del.icio.us TRACK TOP
By Eric Downing | Filed in Cryptocurrency, Useful | No comments yet.

Bitcoin is booming and everyone wants a piece of the action. I have been using the Freebitco.in faucet site to get my piece of the action. The nice thing about the Freebitcoin.com site is that they are backed by mining that is creating value for their site and powering the faucet. There is also a multiplier game, Betting and contests that can gain you extra satoshi.

The site has a faucet that you can claim from hourly. This is not huge profit. I believe it is now about 2 satoshi per hour if you have not played the multiplier game which can increase you share. Once you have over 30,000 satoshi, they start paying you interest.

The multiply HI/LO game can gain you a profit, but can also lose quite a bit quickly if you are not careful. There is an automated version of the multiply game where you can set the odds, how much you want to bet and how many rolls to take. You can also set limits on how much you can lose, but I wouldn’t start playing until you had a sizeable amount of satoshis to withstand a loss run.

There is also a premium token being added to the site. This is to increase the rewards for being on the site. The token is the FUN token. Depending on the amount of FUN tokens held you can gain extra spins, increased interest and even some cashback opportunities.

Now 2 satoshis per hour doesn’t sound like much but if Bitcoin takes off like it has it is a easy free way to collect some of those gains for yourself.

So check it out at and join @ Freebitco.in

Feel free to ask any questions about the site.

How to report Scam calls

digg del.icio.us TRACK TOP
By Eric Downing | Filed in Useful | No comments yet.

Having a phone number leaves you open to scam calls. To report these calls you can go to the following website and let them know about the call.

https://reportfraud.ftc.gov/#/

On this site you can report if the caller asked for money, gave a name and when they called. Not sure what the full end result of calls can be but I feel it is my drop in the bucket to help combat these calls.

Some scams that I have received:

  • Social Security Number has been suspend
    • The government would not call to suspend your SSN.
    • You should never give your personal information to a random call
  • Police association requesting money
  • You’ve won a prize but have to pay to collect
  • Large sum of money but you need to pay some fees to receive it

Python: Convert int to String for output

digg del.icio.us TRACK TOP
By Eric Downing | Filed in Python | No comments yet.

I was creating a new program and wanted to generate an output string that showed the count of certain types of entries. When I attempted the simple output to get “Number of rows: 5” by using the following:

Incorrect call

rowcount = 100
print("Number of Rows: " + rowcount) 

I got the following error “TypeError: can only concatenate str (not “int”) to str” . This meant that the type of rowcount was an int but needed to be a string. There is a handy function call that takes the int and converts it to a string. The function is str() and it converts the int argument to a string value.

rowcount = 100
print("Number of Rows: " + str(rowcount))

There are other ways to output a number such as ‘%s’, format and f-string. But for my purposes. ‘str’ gave me the debugging info I needed.

Tags: