# Questions tagged [numpy]

24110 questions

1

votes

1

answer

21

Views

### Construction of Python ndarray which uses strings as indices

I am most certainly a newbie when it comes to python. About 1 week into learning it and realize there are a lot of tools here I should be using. One of them is numpy.darray. However i am having difficuly applying it to my problem.
I have the following array data structure: string -> value. That is...

0

votes

0

answer

32

Views

### Python3 - TypeError: 'numpy.float64' object is not iterable

I want to make a plot that shows the missclassification error versus de K neighbors using KNN.
This the code i've built for that:
# creating odd list of K for KNN
myList = list(range(1,50))
# subsetting just the odd ones
neighbors = filter(lambda x: x % 2 != 0, myList)
# empty list that will hold cv...

-2

votes

0

answer

20

Views

### Crop image from center in openCV using python

I am using openCV in python to auto detect objects in image and then crop it.But this cropping i need from center instead of starting top left corner in openCV. the following line of code cropping from top left corner.
image = image[minY:maxY,minX:maxX]

1

votes

1

answer

917

Views

### Splitting numpy array according bool

I have an array like
a = np.array[ 4, 9, 3, 1, 6, 4, 7, 4, 2]
and a boolean array (so that's a mask) of same size like
boo = np.array[ True, True, False, False, True, True, True, False, True]
(boo can also start with a False as first entry...)
Now I want to split a into new arrays with 2 condition...

1

votes

2

answer

401

Views

### Is it possible to use einsum to transpose everything?

Ok, I know how to transpose a matrix, with for instance:
A = np.arange(25).reshape(5, 5)
print A
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
A.T
array([[ 0, 5, 10, 15, 20],
[ 1, 6, 11, 16, 21],
[ 2, 7, 12, 17, 22],
[ 3, 8,...

1

votes

2

answer

459

Views

### How can I put a string to an array

I want to put a string to an array location but I get an error:
ValueError: could not convert string to float
My code is the following:
k = np.ceil(99/8)
rs = np.zeros((int(k), 10))
for i in range(0, int(k)):
rs[i, 0] = "FREQ"
for j in range(1,9):
rs[i, j] = rs_imp[8*k+j, 0]

1

votes

3

answer

2.2k

Views

### Flip and rotate numpy array

Is there a faster way of flipping and rotating an array in numpy? For example, rotating one time clockwise and then flipping?
import numpy as np
a = np.arange(0,10)
b = np.arange(-11,-1)
ar = np.array([a,b])
print ar
print ar.shape
ar = np.rot90(ar, 3)
print np.fliplr(ar)
print ar.shape
Output:
[[...

1

votes

1

answer

1.7k

Views

### Convert a list of dictionaries to numpy matrix? [duplicate]

This question already has an answer here:
Convert list of dictionaries to a pandas DataFrame
4 answers
Let the given dictionaries are
d = [{'a':1,'b':4},{'b':2}]
So basically I want a matrix like this
| 'a' | 'b' |
_______________
| 1 | 4 |
| na | 2 |
How can I efficiently achieve this ?

0

votes

0

answer

9

Views

### Compact form of python first and second elements of array

I have defined two arrays:
a=np.array([[2,3,4],[5,6,7],[8,9,10]])
b=np.array([-1,-2])
and created a third one:
x=np.asarray([[x - a/2, x + a/2] for x in b])
Now, I have defined two variables
u,v = x[:,0], x[:,1]
My question is extremely simple: is there a way to define those variables without the c...

3

votes

5

answer

40

Views

### Data-frame filter according to a condition

I have a data-frame like
colA colB colC
A B C
A D C
B B E
A D C
C B C
I want to filter them in a priority like this:
If colC == E then return E, after that check colB == D return D otherwise return colA
The output is
colA colB colC final
A B...

0

votes

1

answer

17

Views

### How to sum annual data in a csv of daily data with many years of data

Using a csv of daily data I am trying to find the annual total precipitation for every year. My csv has many years worth of data.
I've tried AnnualMaxPRCP=df3.resample('AS').sum() but this also sums my year column. How can I sum just the Precip column?
df = pd.read_csv("Current2.csv")
df['Date']=pd...

1

votes

1

answer

68

Views

### Intuitive understanding of Numpy nd-array

So I've read the manual - but the structure still comes confusing to me. Specifically, what is the relationship between:
nd-array and Python list?
nd-array and Python dictionary?
nd-array and Pandas DataFrame?
nd-arrays and Numpy "structured arrays"?
Also, is nd-array just like a sequence of lists?...

1

votes

2

answer

1.3k

Views

### How to compute moving (or rolling, if you will) percentile/quantile for a 1d array in numpy?

In pandas, we have pd.rolling_quantile(). And in numpy, we have np.percentile(), but I'm not sure how to do the rolling/moving version of it.
To explain what I meant by moving/rolling percentile/quantile:
Given array [1, 5, 7, 2, 4, 6, 9, 3, 8, 10], the moving quantile 0.5 (i.e. moving percentile 50...

1

votes

1

answer

847

Views

### Converting string of “bytes” to numpy array of uint8

I'm using a API that returns RGB data as strings(*) (for instance I get 'ABC' for [65, 66, 67]. Is there a way to have this directly converted to a numpy unint8 array without an explicit comprehension with ord()? Since this is picture data, I could be processing several million bytes, so any shortcu...

1

votes

2

answer

851

Views

### calculate distance of 2 list of points in numpy

I have 2 lists of points as numpy.ndarray, each row is the coordinate of a point, like:
a = np.array([[1,0,0],[0,1,0],[0,0,1]])
b = np.array([[1,1,0],[0,1,1],[1,0,1]])
Here I want to calculate the euclidean distance between all pairs of points in the 2 lists, for each point p_a in a, I want to calcu...

0

votes

0

answer

26

Views

### Vectorizing a recurrence relation for-loop in python

I have a recurrent relation that looks like the following:
f_0 = s_0
f_1 = alpha * s_1 + (1 - alpha) * f_0
f_2 = alpha * s_2 + (1 - alpha) * f_1
...
f_n = alpha * s_n + (1 - alpha) * f_n-1
this is easy enough to code up as a for-loop (example code)
import numpy as np
## values for example only ##
s...

1

votes

2

answer

39

Views

### RAM usage after importing numpy in python 3.7.2

I run conda 4.6.3 with python 3.7.2 win32. In python, when I import numpy, i see the RAM usage increase by 80MB. Since I am using multiprocessing, I wonder if this is normal and if there is anyway to avoid this RAM overhead? Please see below all the versions from relevant packages (from conda list):...

1

votes

1

answer

385

Views

### how to declare numpy.bool in cython?

According the offical doc:
http://docs.cython.org/en/latest/src/tutorial/numpy.html
we should "ctypedef" a corresponding compile-time type,but I tried several methods to deal with numpy.bool. It's still wrong.
(1)
DTYPE2 = np.bool
ctypedef np.bool_t DTYPE2_t
raise:
'bool_t' is not a type identifier...

1

votes

1

answer

129

Views

### python - save dict to .npy format

I have a dictand I associate an array to each key (the key itself is a number).
Minimal example:
import numpy as np
data = {}
data[2.5] = np.array([np.array([1,2,3,4]), np.array([5,6,7,8])])
Then I save the dict:
np.save('file.npy', data)
and then reload it:
datanew = np.load('file.npy')
--
Now, in...

1

votes

1

answer

8.5k

Views

### Cannot uninstall 'numpy'

I've been trying to install astropy and at the end of the installation I get this message:
Cannot uninstall 'numpy'. It is a distutils installed project and thus
we cannot accurately determine which files belong to it which would
lead to only a partial uninstall.
I have tried: pip uninstall numpy a...

1

votes

1

answer

284

Views

### Expected 2-D array, got 1-D array instead

from sklearn import MinMaxScaler, StandardScaler
import numpy as np
a = ([1,2,3],[4,5,6])
stan = StandardScaler()
mima = MinMaxScaler()
stan.fit_tranform(a)
mima.fit_transform(a)
results after runnin stan and mima
array([[-1., -1., -1.],
[ 1., 1., 1.]])
array([[0., 0., 0.],
[1., 1., 1.]])
However,...

1

votes

1

answer

301

Views

### Are Numpy arrays hashable?

I've read that numpy arrays are hashable which means it is immutable but I'm able to change it's values so what does it exactly mean by being hashable?
c=pd.Series('a',index=range(6))
c
Out[276]:
0 a
1 a
2 a
3 a
4 a
5 a
dtype: object
This doesn't give me error then why it gives er...

-1

votes

2

answer

20

Views

### How to run numpy log in a class

I'm really new to writing python classes. Basically, I'm trying to write a class that will be used in an sklearn pipeline. The class needs to add two attributes to an existing data frame by modifying some existing attributes.
The new attributes will be a log transformation of an existing column, an...

1

votes

0

answer

10

Views

### pandas astype python bool instead of numpy.bool_

I need to convert a pandas dataframe to a JSON object.
However
json.dumps(df.to_dict(orient='records'))
fails as the boolean columns are not JSON serializable since they are of type numpy.bool_. Now I've tried df['boolCol'] = df['boolCol'].astype(bool) but that still leaves the type of the fields a...

1

votes

1

answer

152

Views

### PyArray_SimpleNewFromData

So I am trying to write a C function that accepts a numpy array object, extracts the data, does some manipulations and returns another c array as a numpy array object. Everything works seamlessly and I use python wrappers which help easy manipulation on the python side. However, I am facing a memory...

1

votes

2

answer

44

Views

### Python Pre-Define Function

In my code I use the following structure to avoid conditions in a for loop:
if patch_type == "zeros":
patch_fct = np.zeros
elif patch_type == "ones":
patch_fct = np.ones
elif patch_type == "rand":
patch_fct = np.random.random_sample
else:
raise "Error"
for k in range(10**9):
m, n = comp_size()
bla =...

1

votes

2

answer

32

Views

### Display extracted feature vector from trained layer of the model as an image

I am using Transfer learning for recognizing objects. I used trained VGG16 model as the base model and added my classifier on top of it using Keras. I then trained the model on my data, the model works well. I want to see the feature generated by the intermediate layers of the model for the given d...

1

votes

1

answer

79

Views

### Keras: how to get predicted labels for more than two classes

I implemented an image classifier in Keras, using the TensorFlow backend. With a dataset with two output classes, I checked the predicted labels as follows:
if result[0][0] == 1:
prediction ='adathodai'
else:
prediction ='thamarathtai'
Full code link:
here
With three classes, I get [[0. 0. 1.]]...

1

votes

2

answer

31

Views

### Generating an NxM array of uniformly distributed random numbers over a stated interval (not [0,1)) in numpy

I am aware of the numpy.random.rand() command, however there doesn't seem to be any variables allowing you to adjust the uniform interval in which the numbers are chosen to something other than [0,1).
I considered using a for loop i.e. initiating a zero array of the needed size, and using numpy.rand...

1

votes

2

answer

34

Views

### Rescale matrix by summating over pixels

Good day everyone.
Is there a quick way to rescale a matrix by simply adding adjacent pixels?
So for a X=N*M matrix you get a Y=(N/n) *(N/m) where n * m is the area I should add the pixel in.
I've been doing that manually (via script) but I think there has to be somewhere a way to do it.
for i in ra...

1

votes

3

answer

113

Views

### Python / numpy: Remove empty (zeroes) border of 3D array

I have a 3D numpy array. This can be thought of as an image (to be exact it's values of field points). I want to remove the border (0 values, note that there are negative values possible) in all dimensions. The restriction is that the dimension remains the same for all molecules, eg. I only want to...

1

votes

3

answer

30

Views

### How to add masking noise to numpy 2-D matrix in a vectorized manner?

I have a numpy 2-D array X with shape (n_samples, n_features). I want to apply masking noise to each sample i.e. each row. Basically, for each row entry, I want to randomly select a fraction frac of the total n_features elements and set them to 0.
I have vectorized the inner part of the loop till no...

1

votes

2

answer

27

Views

### Incrementing a python list by an amount mod x_i, where x_i depends on the position

I have two lists of equal length.
I am incrementing list1 by an amount, but each entry has to be modded by the corresponding entry in list2.
What I am doing is:
for ii in range(len(list1)):
list1[ii] = (list1[ii]+val) % list2[ii]
Is there a better way (maybe using numpy)? I could not find any. These...

0

votes

2

answer

45

Views

### how to append an array into list

I have a problem to append my array that created from numpy library. Here is my python's code
# to get array's column name
columnData = [x[0] for x in curHeader.description]
# to get data result
rData = curHeader.fetchall()
# loop data
for i in rData:
arrayData = np.asarray(dict(zip(columnData, i)))...

2

votes

0

answer

14

Views

### Find the number of times a combination occurs in a numpy 2D array

I have a 2D numpy array, and I want a function operating on col1 and col2 of the array, If 'M' is the number of unique values from col1 and 'N' is the number of unique values from col2, then the output 1D array will have size (M * N) For example, suppose there are 3 unique values in col1: A1, A2 and...

0

votes

1

answer

13

Views

### Getting strange results from numpy argsort

I've got (what seems to me) a fairly clear-cut example with numpy argsort where it is producing odd results. If I create an example array of characters:
letters = np.array([['b','a','c'],
['c','a','b'],
['b','c','a']]).astype(str)
I'm then looking to sort along the rows (and to retain the sorting se...

1

votes

0

answer

16

Views

### Python multidimensional array indexing explanation

Could someone please explain to me what is happening here? I understand what is happening here: https://docs.scipy.org/doc/numpy-1.15.0/user/basics.indexing.html#index-arrays, but do not understand this piece of code.
import numpy as np
y = np.zeros((3,3))
y = y.astype(np.int16)
y[1,1] = 1
x = np.on...

1

votes

3

answer

44

Views

### Updating variable name within a for loop while performing calculations

Whenever I stumble across some sort of calculation in Python, I tend to do go for an unpythonic approach because I am not too familiar with the language:
import pandas as pd
import numpy as np
v = 8
gf = 2.5
data_a1 = np.random.randint(5, 10, 21)
data_a2 = np.random.randint(5, 10, 21)...

5

votes

1

answer

38

Views

### Advanced slicing. Higher order slicing/selecting

I'm confused about the semantics of numpy advanced slicing in some code I found in our codebase today. Let me start out with an example:
# example boolean matrix
a = np.random.rand(5, 5) > 0.5
# Outputs
array([[ True, False, True, True, False],
[ True, True, False, True, False],
[False, True,...

0

votes

0

answer

7

Views

### specify dtypes when saving pandas dataframe to a binary file

I have a pandas DataFrame I want to write to a binary file, however the df contains mixed dtypes and ints. If I used df.values.tofile() I cannot specify different dtypes (even when specifying astype('f4, f4, i4, i4').tofile() in below example). Workaround at the moment is to use struct but is very s...