Natural Language Processing
A practical example of how to build several Machine Translation models in Python and Tensorflow
We will build a deep neural network that functions as part of an end-to-end machine translation pipeline. The completed pipeline will accept English text as input and return the French translation. For our model, we will use an English and French sample of sentences. We will load the following libraries:
import collectionsimport helper
import numpy as npfrom keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
from keras.models import Model
from keras.layers import GRU, Input, Dense, TimeDistributed, Activation, RepeatVector, Bidirectional
from keras.layers.embeddings import Embedding
from keras.optimizers import Adam
from keras.losses import sparse_categorical_crossentropy
Where helper.py is:
import osdef load_data(path)…
A tutorial of how you can make interactive maps with folium and pandas
Folium provides a python interface for leaflet.js. Leaflet.js is a Javascript library for interactive maps and can be useful to know on its own. The benefit of using this library via Folium is that Folium makes it very easy to use from within a Jupyter Notebook and to access your python data structures (e.g. Pandas DataFrames). The documentation for Folium can be found on its official website. For this tutorial, we will work with the Baltimore Arrest Data.
Interactive Map with Baltimore Arrest Data
The map will include the following dimensions:
- Age: In the form of
isAdultwhere will be apop-upin every observation - Race…
A walk-through example of Transfer Learning on Images for Classification problems
In this tutorial, we will provide you an example of how you can build a powerful neural network model to classify images of cats and dogs using transfer learning by considering as base model a pre-trained model trained on ImageNet, and then we will train additional new layers for our cats and dogs classification model.
The Data
We will work with a sample of 600 images from the Dogs vs Cats dataset, which was used for a 2013 Kaggle competition.
import tensorflow as tf
from tensorflow.keras.models import Sequential, Model, load_model
from tensorflow.keras.layers import Input, Conv2D, MaxPooling2D, Dense, Flatten, Dropout
import numpy as np
import os
import…Did you know that Medium gives bonuses?
Today, when I visited the “Medium Partner Program” to see my traffic and revenues, I saw this message:
How to apply an algorithmic trading strategy that takes an advantage of big whales movements
In the cryptocurrency market, the big whales play a key role. These movements affect significantly the market. Take for example a well-known big whale, Elon Musk. Whenever he trades a big amount of Bitcoins, the market follows him. In this post, we will try to build an algorithmic trading strategy that takes an advantage of big whales' movements.
The Scenario
We can detect the whales from the per-minute volume. When there is a significant spike in the volumes, it means that there is an anomaly and most probably is due to the trades of a big investor, the so-called “whale”. Usually, the…
Example of Programming Quizzes in Python
It is common to be asked to build some functions in Python (or any other programming language) during interviews in order to assess your coding style, your skills and your way of thinking. Let’s provide some examples:
Assume that we have a sequence of N rolling Die results, taking values from 1 to 6. Assuming that your input will be a sequence of results like “56611166626634416” build the following functions:
Question 1
How many times did it occur in the trial, that exactly two 6s were rolled after each other? …
We have started a series of articles on tips and tricks for data scientists (in Python and R). In case you have missed:
R
1.How To Remove The Correlated Variables From A Data Frame
When we build predictive models, we use to remove the high correlated variables (multi-collinearity). The point is to keep on of the two correlated variables. Let’s see how we can do it in R by taking as an example the independent variables of the iris dataset.
Get the correlation matrix of the IVs of iris dataset:
df<-iris[, c(1:4)] cor(df)Output:
Sepal.Length Sepal.Width Petal.Length Petal.Width
Sepal.Length 1.0000000 -0.1175698…A walk-through example of how you can get a “Sentiment Score” for each word by applying the Naive Bayes theory
We have provided an example of how to get a sentiment score for words in Python based on ratio frequency. For this example, we will work with the Naive Bayes approach taking into consideration a Twitter dataset that comes with NLTK which has been manually annotated. The sample dataset from NLTK is separated into positive and negative tweets. It contains 5000 positive tweets and 5000 negative tweets exactly.
Positive and Negative Probability of a Word
We have provided an example of Naive Bayes Classification where we explain the theory. …
A walk-through tutorial of how you can apply Naive Bayes Classification in NLP tasks from scratch.
In this tutorial, we will get the Bayesian Score of each word as well as of the whole Subject Line. The score will indicate the chance of a Subject Line and/or token being “spam”. You can find the dataset here. We have used the same dataset, in the Email Spam Detector Tutorial, so feel free to compare the Bayesian approach with the Logistic Regression.
Load the libraries
import pandas as pd
import numpy as np
import re from collections
import Counter import stringTheory and Formulas
So how do you train a Naive Bayes classifier?
- The first part of training a naive Bayes…
How to get the Association Rules in Python
We have provided a tutorial of Market Basket Analysis in Python working with the mlxtend library. Today, we will provide an example of how you can get the association rules from scratch. Let's recall the 3 most common association rules:
Association Rules
Given a set of transactions, find rules that will predict the occurrence of an item based on the occurrences of other items in the transaction. For example, we can extract information on purchasing behavior like “ If someone buys beer and sausage, then is likely to buy mustard with high probability “
Let’s define the main Associaton Rules:
Support
It calculates…
George Pipis
Data Scientist @ Persado | Co-founder of the Data Science blog: https://predictivehacks.com/
