Decimal Data Type Loosing Accuracy During Calculations?
Dec 24, 2011
I have to display all the digits of 2^90. This is within the reach of the Decimal format which can display up to 2^96-1. The Decimal format stops displaying after the 15th digit.The real answer for 2^90 is : 12 379 400 392 853 802 274 899 124 224 (29 digits)
Here is what I do:
Dim a As Decimal = 2^90
TextBox1.Text = a
[code].....
View 4 Replies
ADVERTISEMENT
Feb 7, 2012
I am pulling data from a local MSSQL database using a stored procedure, then send the data to a web service. Every part of the component works well except for a pesky problem will a DBNull being returned from the DataRow field, when I know that the field is not null and has valid data. The database and associated INSERT statements that add to this field are designed to not allow NULL entries. When I debug break the program on or right before the line that throws the error I see that the field has valid data for the current row.If I add a null check to the code (as below) the operation continues as normal:
Dim dataResultsTable = Me.myViewTableAdapter.GetData(int)
For Each myDataRow In dataResultsTable.Rows
If worker.CancellationPending Then
[code]....
Why is the read operation returning DBNull instead of the data in the database?
EDIT: Just to be clear, the operation does return the proper data, but the data is not being saved into the variable.
View 2 Replies
Sep 21, 2010
What's the deal with those 23 unused bits in the Decimal data type? You know, there's the 96-bit integer portion, then 16 unused bits, then 8 bits for the exponent (which can only be 0 through 28, so doesn't even use all 8 bits), then 7 more unused bits, then the sign bit [URL]...
View 1 Replies
Jul 2, 2009
In the discussion in this thread: [URL] it was reveiled to me that vb.net, using double, could only calculate values up to 18 digits after decimal.... so I started thinking and came up with this class
Public Class LargeNumber
Private pNumber As String
Public Sub New()
[CODE].............
I want to split the string into a series of doubles.... and than combine them into a large number.... The size of each double component should be 18 digits on a 32 bit computer... so that I will have room to add the last carrier...
View 3 Replies
Nov 15, 2011
my application has a data table, one of the column set as decimal type to store number, data is read from a text file and store into the data table, then display in a datagridview, I found when that number is only 1 digit, that cell displays nothing (I think no number is being store in that cell), if I type a 1 digit number into that cell is fine. for example,
1.1 [ok]
12 [ok]
0.15 [ok]
4 [not display]
9 [not display]
-23.8 [ok]
so which data type I should use on this column, in order to display all this possible numbers
View 5 Replies
Aug 5, 2009
Any way (outside of writing rounding routines) to restrict the number of places after the decimal in a Decimal data-type variable?
View 3 Replies
Dec 5, 2010
I have a Visual Basic application that I use to access an SQL 2008 database. When I add a record to my table everything appears to be fine. As long as I keep visual studio running the data remains in the database. But if I exit visual studio and then reopen visual studio the data in the table is gone. I can reenter the data and again everything look fine. When I created this application I selected my SQL Server database from the Add New Item dialog. I selected service-based database. Instead of creating the database from the server explorer. Are there settings I need to change when a database is created this way?
View 1 Replies
Jun 9, 2011
I have a bunch of calculations, which will run down for x amount of years or months, specified by the user. These will be in a table (datagridview or something similar), which I will need to access to create reports. So basically calculating the data, then dragging various values at intervals. So it could be any cell at random or patterned. An example of this would be watching how much tax a person paid per year, and how much their income was per year. A pretty basic example. So there would be two columns, and x amount of rows depending on what the user wants to see, in this example, the next 48 months, in months not years. The user then decides to take every 12th value and putting that straight into a report, showing their expected schedule for tax and income.What sort of table structure would be the best to use? The table should be created automatically each time the event (for examples sake a button) is clicked. I'm not sure what I need the event to be based on yet. Most likely the form loading, or when the user tabs onto a new area. Should I use a datagrid view?
View 8 Replies
Sep 15, 2010
I have a Job Register Table and that table doesn't have any records.
This is my LINQ code:
Dim QRecordCount = (From LC In CntxtJobDetails.JobRegistrations _
Where LC.JobCode <> 0 _
[code].....
View 1 Replies
Nov 18, 2010
I need a little guidance in the right direction to help with using data from a dataset in calculations. I've watched the "How Do I" Series and it was very helpful in setting up the relations and tables. I have setup a couple tables to hold values that won't
be updated and the user will never see. Lets say I have 5 columns, ColmnA to ColumnE, and the row count is variable. What I can't seem to find is an easy way to loop through using the variables. As an example, Lets say the equation would be outputted to a
textbox:
textbox1.text=row1(ColumnA*ColumnB+ColumnC^ColumnD-ColumnE)+row2(ColumnA*ColumnB+ColumnC^ColumnD-ColumnE)+etc to end of rowcount
Does anyone know if a good tutorial to point me in the right direction or a code snippet that may get me started to follow the logic? I'm using VS2010Express with a SQLExpress database if that helps.
View 8 Replies
Jun 9, 2011
I'm writing a financial calculator. I know that my calculations are more precise when Decimal format is used. But when I calculate 314/365, the answer given by VB is 0.8602814581, where as the real answer is 0.8602739726. This ends up affecting my calculations quite a bit.
View 7 Replies
Feb 26, 2012
For each column in data table i want to add parameter with same data type as column data type. But looks it's not that simple.
Dim cmd As New SqlCommand
Dim dType As New System.Data.SqlDbType
For Each cl As DataColumn In DataTable1.Columns
[CODE]...
How can I do this?
View 4 Replies
Jan 12, 2011
I have an Access database that has thousands of records. The fields in the database are EMPID, FIRST, LAST, JOBTYPE and PAGES. I would like to know how to loop through the database and for each individual EMPID to calculate the sum of the PAGES foreach JOBTYPE. There are four JOBTYPE's: "E", "K", "O" and "R".
Her is some code:
Dim
WebAllEmployeeQuery As
[code].....
View 12 Replies
Jun 18, 2012
I have a unbound datagridview which i use to enter data. I would like to make calculations using individual cell data as follows:-
Datagridview1.rows(0).cells(1).value = datagridview1.rows(1).cells(1).value + datagridview1.rows(1).cells(2).value
This works fine but if im using lots of cells it can be quite confusing .I was wondering if there was a way that i can give a cell a unique name so i can be easy for me to identify. Using the above example it could read TOTAL = JAN TOTAL + FEB TOTAL
View 4 Replies
Jun 27, 2012
What is the cleanest, most readable way to String.Format a decimal with the following criteria
start with a sign symbol (+ or -)
a fixed number of fraction digits
no decimal separator
right aligned
pre-padded with "0"'s
View 3 Replies
Mar 11, 2012
how to get the time using VB.Net.It has some useful information in addition to the time such as Latitude and Longitude and on what date Daylight Saving starts as in "Spring forward, fall back".This is different for different countries.
View 1 Replies
Jan 13, 2011
I have an Access database that has thousands of records. The fields in the database are EMPID, FIRST, LAST, JOBTYPE and PAGES. I would like to know how to loop through the data table and for each individual EMPID to calculate the sum of the PAGES for each JOBTYPE.There are four JOBTYPE's: "E", "K", "O" and "R".Here is some
Dim WebAllEmployeeQuery As String = _
"SELECT ELPallEmployees.[EMPID], ELPallEmployees.[FIRST], ELPallEmployees.[Last], ELPallEmployees.[JOBTYPE],
[code].....
View 2 Replies
Jul 16, 2010
Does anyone know how to store data from a RTD server into a array for calculations...?
View 2 Replies
Jun 6, 2011
I have a piece of code im working on and is stuck on Select Sum statement which keeps returning a Conversion from type DBNull to type Decimal is invalid. Below is the code and Im working with a Access Database. Everything works good if the Batch Number exists and its when the Batch Number does not exist.
[Code]...
View 10 Replies
Jun 21, 2011
I'm trying to detect if a value is a decimal type and insert it into a DB. If the value is a decimal, I want to continue with the inserting. If the value is not a decimal I want to insert "0" as a value to the DB.
For this I created a function seen below... What is wrong with it?
Function IfNotDeci(ByVal Arvo) As String
If Arvo is Decimal = True Then
Return Arvo
[Code]....
The problem is on the first "If". How to know if the value is a decimal?
(I need this code because my values are read from different excel-files that are not always correct. So if a human didn't write a decimal value to a cell, the program should not respond with an error, but just insert a "0".)
View 3 Replies
Feb 2, 2010
I have a database that dynamically loads from Access into a datagridview. I have to enter data manually into one column in this datagridview. I need to be able to perform calculations to this data, for example I need to sum the data in this column, sqaure it and substract it from individual sqaures of the data in the cells. I should be able to sum it sum(22,33,44,53,67,23)^2-sum(22^2,33^2,...23^2). The number of columns keep changing depending on the database I choose to load, so I need to code this dynamically too. I am not really a VB person. I did the same in excel using Visual Basic and I am trying to do it in this Visual Basic environment.I built a tool in excel(VBA) that do some statistical calculations and data analysis and I am trying to build the same in Visual Basic.
View 10 Replies
Jul 14, 2009
Can I Post a Windows Application that uses Excel for live data calculations? Basically from the Windows app, a user makes a menu selection and which in turn opens an excel file in memory using it for calculations. These files are put together by different groups and formatted for the Windows app to build interfaces based on each file.I also accompany an .mdf file as well. Is all this possible for customers to access and run this with Windows Azure.
View 2 Replies
Aug 5, 2010
I am attempting to convert a DBNull value to a decimal (0, 0.0, 0.00 etc).
My current code is If row.Item("Name of row") Is DBNull.Value Then NameOfRow = Convert.ToDecimal(0.0) End If When i run my app i pass in the row to convert. Every other datatype works/runs as expected except Decimal. It runs with the abbreviated code below:
For each row..in dataset.rows Object = row.table("Table").Item("Name of row")Next The Object.... line causes an error. The error is DBNull' to type 'Decimal' is not valid.... Ive tried many attempts to overcome this but cant get this to work. Could anyone guide me?
View 4 Replies
Jan 1, 2012
here is a very important matter to be corrected as soon as possible. generally,type double can display an answer correct to more than 20 decimal places. yes,it works fine until here except for exponential system. whenever i try to find a decimal raised to the power of a proper fraction, i get a correct answer but only upto 13 decimal places. that is not fair at all! it is important to get as high precision as possible. consider this example:-
[code...]
i want that number of correct decimal places or approximation,which a windows 7 calculator has. i believe that this is someway easy to get rid of. please support.it is important for programs like calculator,worksheets,databases,etc.
View 12 Replies
Apr 20, 2010
When I click on a header (Price) to sort the rows by that column, I get an error Object must be of type decimal.
I'm thinking that I'm encountering a new row without any values that's why this error is being generated.
I'm not sure how to intercept this error or fix it for that matter....
View 7 Replies
Oct 21, 2010
I am using Streamreader to read a text file containing data such as the following:
IN;SC;PU;PU;SP1;LT;VS10
IN;SC;PU;PU;SP1;LT;VS10
PW5;PU4179,27448;PD4179,29951;PD3244,29951;PD3244,30286;PD5494,30286;PD5494,29951;PD4555,29951;PD4555,27448;PD4179,27448;
The final goal is to use the numbers next to the PU and PD characters to perform specific calculations.
Essentially, I think the code needs to do the following:
1. Read each block of characters ending in a ";"
2. If the block commences with anything other than PU or PD, discard it
3. Extract the numbers in such a way that calculations can be performed. The following is an example:
x y
PU4179,27448;
PD4179,29951;
[Code]....
Essentially, there will be hundreds of these rows and I will need to sum up the deltas so that a specific formula can be applied.
I have tried doing this a number of ways but each seem very inefficient (using text boxes to store/swap data and creating additinal text files are 2 methods I have tried) and so I'm looking for some direction.
View 3 Replies
Mar 12, 2012
Which is the most efficient and speedy data structure in order to carry out calculations on it?. I have seen, collections, arrays, list arrays, databases and more. I would prefer to write just straight forward code that will be effective and which I can still understand. What I would like to do initially is have the ability to add new draws to the end of the file, edit the individual files and delete lines if required.
File would then be reformatted to show the corrections made. Some kind of data entry window is envisioned. As far as number crunching goes, I would like to make counts of each number in each of the columns , calculating averages, times since last drawn and so on.
Without being too long winded, I would eventually like to find triple combinations and maybe use them as predictions. So there you have it. This way I won�t have to go off on the wrong path, on a data structure that may not be able, or be too slow for the tasks at hand.
View 5 Replies
Sep 8, 2010
I want to assign a Value to a Variable of Decimal Type to do a calculation, but when I enter the decimal value (a 28 digits to the right of the decimal point - very long numbers) it gets altered. I give below what I was doing:
Dim MyNumber As Decimal = 0.7777777777777777777777777774
But as soon as I finish typing the number, it becomes 0.77777777777777779 and appears as,
Dim MyNumber As Decimal = 0.77777777777777779
And, when the cursor is kept over the altered number it shows as "Double, Represents a Double - precision floating-point number." How can I get the value I wanted to stay without getting changed? I was able to get this long Decimal value as the result of a calculation and get that appears in a text box in the same program. Is there anything I should change in the "Visual Studio 2008" to get this ?
View 5 Replies
Jun 11, 2012
I need to create a function which have either 1 parameter with a string as data type or 2 parameters with double as data type.Something like the substring method.
View 2 Replies
Jan 28, 2010
Please excuse the newbie question as I started learning VB last week . I wrote the following test code below to add two values entered in two textboxes and display it in another textbox. However I keep getting the exception. "Conversion from string "" to type 'Decimal' is not valid" with the code below and I cannot figure out why . I thought converting using 'CDec' resolves this issue ?
View 3 Replies